[Detector Support]: OpenVino: [GPU] Context was not initialized for 0 device

[Redsandro]

Describe the problem you are having

I can't get the OpenVino detector to work using the default model that comes with Frigate Docker. It crashes with the following error message:

[GPU] Context was not initialized for 0 device

It happens with both AUTO and GPU devices. Not with CPU but that one is undesired for obvious reasons.
This is frigate docker in Ubuntu 24.04 LXC on Proxmox 8.2.
update: The same problem happens in Ubuntu 22.04

Version

0.14.0-beta3

Frigate config file

# https://docs.frigate.video/configuration/detectors
detectors:
  ov:
    type: openvino
    device: AUTO
    model:
      path: /openvino-model/ssdlite_mobilenet_v2.xml

model:
  width: 300
  height: 300
  input_tensor: nhwc
  input_pixel_format: bgr
  labelmap_path: /openvino-model/coco_91cl_bkgr.txt

cameras:
  testcam:
    ffmpeg:
      inputs:
        - path: http://...
          input_args: preset-http-reolink
          roles:
            - record
            - detect
    detect:
      enabled: True
      width: 640
      height: 480
      fps: 5
      max_disappeared: 25
    objects:
      filters:
        cat:
          min_score: 0.5
          threshold: 0.6

database:
  path: /db/frigate.db

ffmpeg:
  hwaccel_args: preset-vaapi

docker-compose file or Docker CLI command

version: "3.7"

volumes:
  db:
    name: frigate-db
    external: true

services:
  frigate:
    image: ghcr.io/blakeblackshear/frigate:0.14.0-beta3
    container_name: frigate
    restart: unless-stopped
    logging:
      options:
        max-size: "10m"
        max-file: "30"
    privileged: false
    shm_size: "99mb"
    devices:
      #- /dev/dri/card0
      #- /dev/dri/card1
      #- /dev/dri/card1:/dev/dri/card0 # does not work
      #- /dev/dri/renderD128
      - /dev/dri
    volumes:
      - ./config.yml:/config/config.yml:ro
      - db:/db
      - /etc/localtime:/etc/localtime:ro
      - /opt/frigate/media:/media/frigate
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    expose:
      - 5000
    ports:
      - "1935:1935" # RTMP feeds
    environment:
      FRIGATE_RTSP_PASSWORD: "secret"

Relevant log output

[2024-07-01 13:54:58] frigate.app                    INFO    : Capture process started for testcam: 426
Process detector:ov:
Traceback (most recent call last):
  File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
    self.run()
  File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/opt/frigate/frigate/object_detection.py", line 102, in run_detector
    object_detector = LocalObjectDetector(detector_config=detector_config)
  File "/opt/frigate/frigate/object_detection.py", line 53, in __init__
    self.detect_api = create_detector(detector_config)
  File "/opt/frigate/frigate/detectors/__init__.py", line 18, in create_detector
    return api(detector_config)
  File "/opt/frigate/frigate/detectors/plugins/openvino.py", line 38, in __init__
    self.interpreter = self.ov_core.compile_model(
  File "/usr/local/lib/python3.9/dist-packages/openvino/runtime/ie_api.py", line 521, in compile_model
    super().compile_model(model, device_name, {} if config is None else config),
RuntimeError: Exception from src/inference/src/cpp/core.cpp:123:
Exception from src/inference/src/dev/plugin.cpp:59:
Check 'contexts.count(device_id)' failed at src/plugins/intel_gpu/src/plugin/plugin.cpp:204:
[GPU] Context was not initialized for 0 device

Operating system

Proxmox

Install method

Docker Compose

Object Detector

OpenVino

Any other information that may be helpful

lspci -v | grep -i vga -A8
# lspci: Unable to load libkmod resources: error -2
# 00:02.0 VGA compatible controller [0300]: Intel Corporation Meteor Lake-P [Intel Arc Graphics] [8086:7d55] (rev 08) (prog-if 00 [VGA controller])
#         DeviceName: Onboard - Video
#         Subsystem: ASUSTeK Computer Inc. Meteor Lake-P [Intel Arc Graphics] [1043:88c8]
#         Flags: bus master, fast devsel, latency 0, IRQ 215, IOMMU group 0
#         Memory at 5017000000 (64-bit, prefetchable) [size=16M]
#         Memory at 4000000000 (64-bit, prefetchable) [size=256M]
#         Expansion ROM at 000c0000 [virtual] [disabled] [size=128K]
#         Capabilities: <access denied>
#         Kernel driver in use: i915


vainfo | grep info
# error: XDG_RUNTIME_DIR is invalid or not set in the environment.
# error: can't connect to X server!
# libva info: VA-API version 1.20.0
# libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
# libva info: Found init function __vaDriverInit_1_20
# libva info: va_openDriver() returns 0
# vainfo: VA-API version: 1.20 (libva 2.12.0)
# vainfo: Driver version: Intel iHD driver for Intel(R) Gen Graphics - 24.1.0 ()
# vainfo: Supported profile and entrypoints


ls -l /dev/dri/
# total 0
# drwxr-xr-x 2 nobody nogroup       80 Jul  1 10:05 by-path
# crw-rw---- 1 nobody nogroup 226,   1 Jul  1 10:05 card1
# crw-rw-rw- 1 nobody nogroup 226, 128 Jul  1 10:05 renderD128


cd
apt install python3.12-venv
python3 -m venv openvino_env
. openvino_env/bin/activate
pip3 install openvino
python3 /usr/share/openvino/samples/python/hello_query_device/hello_query_device.py | grep -A99 GPU\ :
# [ INFO ] GPU :
# [ INFO ]        SUPPORTED_PROPERTIES:
# [ INFO ]                AVAILABLE_DEVICES: 0
# [ INFO ]                RANGE_FOR_ASYNC_INFER_REQUESTS: 1, 2, 1
# [ INFO ]                RANGE_FOR_STREAMS: 1, 2
# [ INFO ]                OPTIMAL_BATCH_SIZE: 1
# [ INFO ]                MAX_BATCH_SIZE: 1
# [ INFO ]                DEVICE_ARCHITECTURE: GPU: vendor=0x8086 arch=v12.71.4
# [ INFO ]                FULL_DEVICE_NAME: Intel(R) Arc(TM) Graphics (iGPU)
# [ INFO ]                DEVICE_UUID: 8680557d080000000002000000000000
# [ INFO ]                DEVICE_LUID: 409a0000499a0000
# [ INFO ]                DEVICE_TYPE: Type.INTEGRATED
# [ INFO ]                DEVICE_GOPS: {<Type: 'float16'>: 9625.599609375, <Type: 'float32'>: 4812.7998046875, <Type: 'int8_t'>: 19251.19921875, <Type: 'uint8_t'>: 19251.19921875}
# [ INFO ]                OPTIMIZATION_CAPABILITIES: FP32, BIN, FP16, INT8, EXPORT_IMPORT
# [ INFO ]                GPU_DEVICE_TOTAL_MEM_SIZE: 94020390912
# [ INFO ]                GPU_UARCH_VERSION: 12.71.4
# [ INFO ]                GPU_EXECUTION_UNITS_COUNT: 128
# [ INFO ]                GPU_MEMORY_STATISTICS: {}
# [ INFO ]                PERF_COUNT: False
# [ INFO ]                MODEL_PRIORITY: Priority.MEDIUM
# [ INFO ]                GPU_HOST_TASK_PRIORITY: Priority.MEDIUM
# [ INFO ]                GPU_QUEUE_PRIORITY: Priority.MEDIUM
# [ INFO ]                GPU_QUEUE_THROTTLE: Priority.MEDIUM
# [ INFO ]                GPU_ENABLE_SDPA_OPTIMIZATION: True
# [ INFO ]                GPU_ENABLE_LOOP_UNROLLING: True
# [ INFO ]                GPU_DISABLE_WINOGRAD_CONVOLUTION: False
# [ INFO ]                CACHE_DIR: 
# [ INFO ]                CACHE_MODE: CacheMode.OPTIMIZE_SPEED
# [ INFO ]                PERFORMANCE_HINT: PerformanceMode.LATENCY
# [ INFO ]                EXECUTION_MODE_HINT: ExecutionMode.PERFORMANCE
# [ INFO ]                COMPILATION_NUM_THREADS: 2
# [ INFO ]                NUM_STREAMS: 1
# [ INFO ]                PERFORMANCE_HINT_NUM_REQUESTS: 0
# [ INFO ]                INFERENCE_PRECISION_HINT: <Type: 'float16'>
# [ INFO ]                ENABLE_CPU_PINNING: False
# [ INFO ]                DEVICE_ID: 0


uname -a
# Linux frigate 6.8.8-2-pve #1 SMP PREEMPT_DYNAMIC PMX 6.8.8-2 (2024-06-24T09:00Z) x86_64 x86_64 x86_64 GNU/Linux

LXC configuration

arch: amd64
cores: 2
features: nesting=1
hostname: frigate
memory: 2048
net0: name=eth0,bridge=vmbr0,firewall=1,hwaddr=BC:24:11:EB:AB:E0,ip=dhcp,type=veth
ostype: ubuntu
rootfs: local-lvm:vm-101-disk-0,size=8G
swap: 2048
unprivileged: 1
lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:1 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file
lxc.mount.entry: /dev/dri dev/dri none bind,optional,create=dir

[NickM-27]

does it work if CPU is used as the device?

[ubawurinna]

OpenVINO 2024 is not yet supported by Ubuntu 24.04
System Requirements

[Redsandro]

does it work if CPU is used as the device?

@NickM-27 yes the CPU device works. There are some error messages in the logs:

• mbind failed: Operation not permitted
• Unable to poll intel GPU stats: Failed to initialize PMU! (Operation not permitted) (dozens)
  But Frigate doesn't crash.

OpenVINO 2024 is not yet supported by Ubuntu 24.04

@ubawurinna seriously? I chose 24.04 on purpose because the repo is called openvino/2024 ubuntu24 main and I assumed ubuntu24 meant I needed to use Ubuntu 24.* container.

I will build a new container using Ubuntu 22.04 and see if this is indeed the problem.

[Redsandro]

@ubawurinna I'm running into the exact same problem with Ubuntu 22.04. 😞

uname -a
# Linux frigate_2204 6.8.8-2-pve #1 SMP PREEMPT_DYNAMIC PMX 6.8.8-2 (2024-06-24T09:00Z) 
# x86_64 x86_64 x86_64 GNU/Linux

lsb_release -a
# Distributor ID:	Ubuntu
# Description:		Ubuntu 22.04.4 LTS
# Release:		22.04
# Codename:		jammy

[ubawurinna]

You can take a look at a similar problem that has been solved here:
#12012

[Redsandro]

@ubawurinna thank you, this is interesting! It looks like the docker image is missing a couple of packages:

docker exec -ti frigate bash
cd $(mktemp -d)

wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.16900.23/intel-igc-core_1.0.16900.23_amd64.deb
wget https://github.com/intel/intel-graphics-compiler/releases/download/igc-1.0.16900.23/intel-igc-opencl_1.0.16900.23_amd64.deb
wget https://github.com/intel/compute-runtime/releases/download/24.22.29735.20/intel-level-zero-gpu_1.3.29735.20_amd64.deb
wget https://github.com/intel/compute-runtime/releases/download/24.22.29735.20/intel-opencl-icd_24.22.29735.20_amd64.deb
wget https://github.com/intel/compute-runtime/releases/download/24.22.29735.20/libigdgmm12_22.3.19_amd64.deb

dpkg -i *.deb

It gives some error messages which I ignored, and Frigate no longer crashes! I wonder if this is being tracked somewhere edit: #12260

Inferencing is a lot faster, but "GPU Usage" remains empty. I'm guessing these is a certain permission missing to detect "GPU Usage". I also cannot use the following command from within the Frigate docker container:

intel_gpu_top 
# Failed to initialize PMU! (Operation not permitted)

The same happens in the parent LXC container so that part is a Proxmox issue. Ergo: I think it works now, at least the GPU inferencing, just not the GPU monitoring itself to be able to verify GPU inferencing due to a Proxmox issue I need to figure out.

[ubawurinna]

Just to know, does it work if you install these packages in the proxmox instance instead of the docker image in the proxmox instance?

[ubawurinna]

In the same message, someone said that updating the Proxmox kernel to version 6.8.8-1 fixed the Openvino problem.

[Redsandro]

In the same message, someone said that updating the Proxmox kernel to version 6.8.8-1 fixed the Openvino problem.

It fixes a different problem, but it still requires the latest Intel Compute Runtime, if I understand correctly. I'm tracking this Compute Runtime dependency here: #12260