[HW Accel Support]: Jetson Orin Nano Not Working

[NicholasJohnson2020]

Describe the problem you are having

Hello - I am using Nvidia Jetson Orin with Ubuntu 22.04, jetpack 6.0, and cuda 12.2.
Currently Frigate works when running just cpu. However when I try any hardware acceleration flag it does not connect and gives following error as seen in logs. Has anyone got frigate with GPU to work on Jetson Orin Nano device? What else is needed?

Version

frigatestable

Frigate config file

mqtt:
  enabled: False

cameras:
  camera1: # <--- this will be changed to your actual camera later
    enabled: True
    ffmpeg:
      hwaccel_args: preset-nvidia-h264
        # - -c:v
        # - h264_cuvid
      inputs:
        - path: rtsp://exp2:horse@10.0.0.212/live #TODO update to camera path
          roles:
            - detect
            - record
      # output_args:
      #   record: -f segment -segment_time 10 -segment_format mp4 -crf 28 -reset_timestamps 1 -strftime 1
    detect:
      width: 640 #update to camera resolution
      height: 480 # update to camera resolution
      fps: 15
    record:
      enabled: True
      retain:
        days: 7
        # mode: all
      
  ip_cam: # <--- this will be changed to your actual camera later
    enabled: False
    ffmpeg:
      hwaccel_args: preset-jetson-h264
      inputs:
        - path: rtsp://admin:admin@10.0.0.238/live #TODO update to camera path
          roles:
            - detect
            - record
    detect:
      width: 640 #update to camera resolution
      height: 480 # update to camera resolution
      fps: 15
    record:
      enabled: True
      retain:
        days: 7
        mode: all

docker-compose file or Docker CLI command

version: "2.4"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable #Jetson nano image jetpack 4.6
    # image: ghcr.io/blakeblackshear/frigate:stable-tensorrt-jp5 #Jetson nano image jetpack 4.6
    shm_size: "64mb" # update for your cameras based on calculation above
    devices:
      - /dev/bus/usb:/dev/bus/usb  # Passes the USB Coral, needs to be modified for other versions
      - /dev/apex_0:/dev/apex_0    # Passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
      - /dev/video11:/dev/video11  # For Raspberry Pi 4B
      - /dev/dri/renderD128:/dev/dri/renderD128 # For intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /home/visionsense/Docker/frigate/config:/config
      - /home/visionsense/Docker/frigate/media:/media/frigate #TODO switch this to external HD for media storage
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "password"
    runtime: nvidia
    deploy:    # <------------- Add this section
      resources:
        reservations:
          devices:
            - driver: nvidia
              #device_ids: ['0'] # this is only needed when using multiple GPUs
              count: 1 # number of GPUs
              capabilities: [gpu]

Relevant log output

frigate  | s6-rc: info: service s6rc-fdholder: starting
frigate  | s6-rc: info: service s6rc-oneshot-runner: starting
frigate  | s6-rc: info: service s6rc-oneshot-runner successfully started
frigate  | s6-rc: info: service fix-attrs: starting
frigate  | s6-rc: info: service s6rc-fdholder successfully started
frigate  | s6-rc: info: service fix-attrs successfully started
frigate  | s6-rc: info: service legacy-cont-init: starting
frigate  | s6-rc: info: service legacy-cont-init successfully started
frigate  | s6-rc: info: service log-prepare: starting
frigate  | s6-rc: info: service log-prepare successfully started
frigate  | s6-rc: info: service nginx-log: starting
frigate  | s6-rc: info: service go2rtc-log: starting
frigate  | s6-rc: info: service frigate-log: starting
frigate  | s6-rc: info: service nginx-log successfully started
frigate  | s6-rc: info: service go2rtc-log successfully started
frigate  | s6-rc: info: service go2rtc: starting
frigate  | s6-rc: info: service frigate-log successfully started
frigate  | s6-rc: info: service go2rtc successfully started
frigate  | s6-rc: info: service go2rtc-healthcheck: starting
frigate  | s6-rc: info: service frigate: starting
frigate  | s6-rc: info: service go2rtc-healthcheck successfully started
frigate  | s6-rc: info: service frigate successfully started
frigate  | s6-rc: info: service nginx: starting
frigate  | s6-rc: info: service nginx successfully started
frigate  | s6-rc: info: service legacy-services: starting
frigate  | s6-rc: info: service legacy-services successfully started
frigate  | 2024-08-14 19:52:43.197164834  [INFO] Starting NGINX...
frigate  | 2024-08-14 19:52:43.197155042  [INFO] Preparing Frigate...
frigate  | 2024-08-14 19:52:43.482458961  [INFO] Starting Frigate...
frigate  | 2024-08-14 19:52:43.485846809  [INFO] Preparing new go2rtc config...
frigate  | 2024-08-14 19:52:45.412364756  [INFO] Starting go2rtc...
frigate  | 2024-08-14 19:52:45.636730455  19:52:45.636 INF go2rtc version 1.8.4 linux/arm64
frigate  | 2024-08-14 19:52:45.638078579  19:52:45.637 INF [rtsp] listen addr=:8554
frigate  | 2024-08-14 19:52:45.638638815  19:52:45.638 INF [webrtc] listen addr=:8555
frigate  | 2024-08-14 19:52:45.638989767  19:52:45.638 INF [api] listen addr=:1984
frigate  | 2024-08-14 19:52:47.977971971  2024/08/14 19:52:47 [error] 158#158: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET /api/version HTTP/1.1", upstream: "http://127.0.0.1:5001version", host: "127.0.0.1:5000"
frigate  | 2024-08-14 19:52:50.802121118  [2024-08-14 19:52:50] frigate.app                    INFO    : Starting Frigate (0.13.2-6476f8a)
frigate  | 2024-08-14 19:52:50.911821470  [2024-08-14 19:52:50] peewee_migrate.logs            INFO    : Starting migrations
frigate  | 2024-08-14 19:52:50.928165306  [2024-08-14 19:52:50] peewee_migrate.logs            INFO    : There is nothing to migrate
frigate  | 2024-08-14 19:52:50.929365844  [2024-08-14 19:52:50] frigate.app                    INFO    : Running database vacuum
frigate  | 2024-08-14 19:52:51.003036821  [2024-08-14 19:52:51] frigate.app                    INFO    : Recording process started: 395
frigate  | 2024-08-14 19:52:51.014887313  [2024-08-14 19:52:51] frigate.app                    INFO    : go2rtc process pid: 98
frigate  | 2024-08-14 19:52:51.059642955  [2024-08-14 19:52:51] detector.cpu                   INFO    : Starting detection process: 404
frigate  | 2024-08-14 19:52:51.070061033  [2024-08-14 19:52:51] frigate.app                    INFO    : Output process started: 406
frigate  | 2024-08-14 19:52:51.071033565  [2024-08-14 19:52:51] frigate.detectors              WARNING : CPU detectors are not recommended and should only be used for testing or for trial purposes.
frigate  | 2024-08-14 19:52:51.091165674  [2024-08-14 19:52:51] frigate.app                    INFO    : Camera processor started for camera1: 411
frigate  | 2024-08-14 19:52:51.091175274  [2024-08-14 19:52:51] frigate.app                    INFO    : Camera processor not started for disabled camera ip_cam
frigate  | 2024-08-14 19:52:51.106997883  [2024-08-14 19:52:51] frigate.app                    INFO    : Capture process started for camera1: 412
frigate  | 2024-08-14 19:52:51.107007228  [2024-08-14 19:52:51] frigate.app                    INFO    : Capture process not started for disabled camera ip_cam
frigate  | 2024-08-14 19:52:53.032310391  [INFO] Starting go2rtc healthcheck service...
frigate  | 2024-08-14 19:52:53.892735930  [2024-08-14 19:52:53] frigate.video                  ERROR   : camera1: Unable to read frames from ffmpeg process.
frigate  | 2024-08-14 19:52:53.894808870  [2024-08-14 19:52:53] frigate.video                  ERROR   : camera1: ffmpeg process is not running. exiting capture thread...
frigate  | 2024-08-14 19:53:11.171823376  [2024-08-14 19:53:11] watchdog.camera1               ERROR   : Ffmpeg process crashed unexpectedly for camera1.
frigate  | 2024-08-14 19:53:11.173333905  [2024-08-14 19:53:11] watchdog.camera1               ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | 2024-08-14 19:53:11.173949374  [2024-08-14 19:53:11] ffmpeg.camera1.detect          ERROR   : Guessed Channel Layout for Input Stream #0.1 : mono
frigate  | 2024-08-14 19:53:11.174074496  [2024-08-14 19:53:11] ffmpeg.camera1.detect          ERROR   : Device creation failed: -12.
frigate  | 2024-08-14 19:53:11.174198883  [2024-08-14 19:53:11] ffmpeg.camera1.detect          ERROR   : [h264 @ 0xaaaac70bf5c0] No device available for decoder: device type cuda needed for codec h264.
frigate  | 2024-08-14 19:53:11.174317350  [2024-08-14 19:53:11] ffmpeg.camera1.detect          ERROR   : Device setup failed for decoder on input stream #0:0 : Cannot allocate memory
frigate  | 2024-08-14 19:53:13.831257357  [2024-08-14 19:53:13] frigate.video                  ERROR   : camera1: Unable to read frames from ffmpeg process.
frigate  | 2024-08-14 19:53:13.831497970  [2024-08-14 19:53:13] frigate.video                  ERROR   : camera1: ffmpeg process is not running. exiting capture thread...
frigate  | 2024-08-14 19:53:21.207433557  [2024-08-14 19:53:21] watchdog.camera1               ERROR   : Ffmpeg process crashed unexpectedly for camera1.
frigate  | 2024-08-14 19:53:21.207724251  [2024-08-14 19:53:21] watchdog.camera1               ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | 2024-08-14 19:53:21.207933536  [2024-08-14 19:53:21] ffmpeg.camera1.detect          ERROR   : Guessed Channel Layout for Input Stream #0.1 : mono
frigate  | 2024-08-14 19:53:21.208073091  [2024-08-14 19:53:21] ffmpeg.camera1.detect          ERROR   : Device creation failed: -12.
frigate  | 2024-08-14 19:53:21.208160452  [2024-08-14 19:53:21] ffmpeg.camera1.detect          ERROR   : [h264 @ 0xaaaae7b8a4c0] No device available for decoder: device type cuda needed for codec h264.
frigate  | 2024-08-14 19:53:21.208448171  [2024-08-14 19:53:21] ffmpeg.camera1.detect          ERROR   : Device setup failed for decoder on input stream #0:0 : Cannot allocate memory
frigate  | 2024-08-14 19:53:23.825623159  [2024-08-14 19:53:23] frigate.video                  ERROR   : camera1: Unable to read frames from ffmpeg process.
frigate  | 2024-08-14 19:53:23.825974687  [2024-08-14 19:53:23] frigate.video                  ERROR   : camera1: ffmpeg process is not running. exiting capture thread...
frigate  | 2024-08-14 19:53:31.246168135  [2024-08-14 19:53:31] watchdog.camera1               ERROR   : Ffmpeg process crashed unexpectedly for camera1.
frigate  | 2024-08-14 19:53:31.247092762  [2024-08-14 19:53:31] watchdog.camera1               ERROR   : The following ffmpeg logs include the last 100 lines prior to exit.
frigate  | 2024-08-14 19:53:31.247099451  [2024-08-14 19:53:31] ffmpeg.camera1.detect          ERROR   : Guessed Channel Layout for Input Stream #0.1 : mono
frigate  | 2024-08-14 19:53:31.247101275  [2024-08-14 19:53:31] ffmpeg.camera1.detect          ERROR   : Device creation failed: -12.
frigate  | 2024-08-14 19:53:31.247114715  [2024-08-14 19:53:31] ffmpeg.camera1.detect          ERROR   : [h264 @ 0xaaaafb534190] No device available for decoder: device type cuda needed for codec h264.
frigate  | 2024-08-14 19:53:31.247149884  [2024-08-14 19:53:31] ffmpeg.camera1.detect          ERROR   : Device setup failed for decoder on input stream #0:0 : Cannot allocate memory
frigate  | 2024-08-14 19:53:33.825298966  [2024-08-14 19:53:33] frigate.video                  ERROR   : camera1: Unable to read frames from ffmpeg process.
frigate  | 2024-08-14 19:53:33.826950842  [2024-08-14 19:53:33] frigate.video                  ERROR   : camera1: ffmpeg process is not running. exiting capture thread...

FFprobe output from your camera

Unsure how to run this

Operating system

Other Linux

Install method

Docker Compose

Network connection

Wireless

Camera make and model

wyze camv3

Any other information that may be helpful

Jetson Orin Nano has aarch64 architecture and is running jetpack 6.0 with ubuntu 22.04

[NickM-27]

what is your docker compose

[NicholasJohnson2020]

Hi Nick, I included my docker compose in my original post. Pasting it below for ease of reference:

version: "2.4"
services:
  frigate:
    container_name: frigate
    privileged: true # this may not be necessary for all setups
    restart: unless-stopped
    image: ghcr.io/blakeblackshear/frigate:stable #Jetson nano image jetpack 4.6
    # image: ghcr.io/blakeblackshear/frigate:stable-tensorrt-jp5 #Jetson nano image jetpack 4.6
    shm_size: "64mb" # update for your cameras based on calculation above
    devices:
      - /dev/bus/usb:/dev/bus/usb  # Passes the USB Coral, needs to be modified for other versions
      - /dev/apex_0:/dev/apex_0    # Passes a PCIe Coral, follow driver instructions here https://coral.ai/docs/m2/get-started/#2a-on-linux
      - /dev/video11:/dev/video11  # For Raspberry Pi 4B
      - /dev/dri/renderD128:/dev/dri/renderD128 # For intel hwaccel, needs to be updated for your hardware
    volumes:
      - /etc/localtime:/etc/localtime:ro
      - /home/visionsense/Docker/frigate/config:/config
      - /home/visionsense/Docker/frigate/media:/media/frigate #TODO switch this to external HD for media storage
      - type: tmpfs # Optional: 1GB of memory, reduces SSD/SD Card wear
        target: /tmp/cache
        tmpfs:
          size: 1000000000
    ports:
      - "5000:5000"
      - "8554:8554" # RTSP feeds
      - "8555:8555/tcp" # WebRTC over tcp
      - "8555:8555/udp" # WebRTC over udp
    environment:
      FRIGATE_RTSP_PASSWORD: "password"
    runtime: nvidia
    deploy:    # <------------- Add this section
      resources:
        reservations:
          devices:
            - driver: nvidia
              #device_ids: ['0'] # this is only needed when using multiple GPUs
              count: 1 # number of GPUs
              capabilities: [gpu]

[EtoTen]

Hey I have the same hardware (Orin AGX on JP5). It is your config that is off (like mine was). Probably missing the "hwaccel_args: preset-jetson-h264" under the camera ffmpeg setting.
Here is my config:

mqtt:
  enabled: false  # Set this to true if you want to use MQTT
  host: mqtt.server.com
  user: mqtt_user
  password: password

detectors:
  tensorrt:
    type: tensorrt
    device: 0 #This is the default, select the first GPU

ffmpeg:
  hwaccel_args:  preset-nvidia-h264

snapshots:
  enabled: true
  retain:
    # Required: Default retention days (default: shown below)
    default: 15
    # Optional: Per object retention days
    objects:
      person: 15
  # Optional: quality of the encoded jpeg, 0-100 (default: shown below)
  quality: 100

model:
  path: /config/model_cache/tensorrt/yolov7-320.trt
  input_tensor: nchw
  input_pixel_format: rgb
  width: 320
  height: 320

record:
  enabled: true
  retain:
    days: 14
    mode: all
  events:
    pre_capture: 15
    post_capture: 15
    retain:
      default: 8
      mode: motion
  export:
    # Optional: Timelapse Output Args (default: shown below).
    # NOTE: The default args are set to fit 24 hours of recording into 1 hour playback.
    # See https://stackoverflow.com/a/58268695 for more info on how these args work.
    # As an example: if you wanted to go from 24 hours to 30 minutes that would be going
    # from 86400 seconds to 1800 seconds which would be 1800 / 86400 = 0.02.
    # The -r (framerate) dictates how smooth the output video is.
    # So the args would be -vf setpts=0.02*PTS -r 30 in that case.
    timelapse_args: -vf setpts=0.00054*PTS -r 25

objects:
  # Optional: list of objects to track from labelmap.txt (default: shown below)
  track:
    - person
    - face
    - license_plate
    - car
    - bicycle
    - motorcycle
    - bus
    - amazon
    - fedex
    - ups
    - cat
    - package
    - dog
    - deer
    - horse
    - sheep
    - cow
    - bear
    - airplane
    - bird
# Include all cameras by default in Birdseye view
birdseye:
  enabled: true
  mode: objects
  width: 1280
  height: 720
cameras:
  RoofPTZ:
    ui:
      order: 1
    ffmpeg:
      hwaccel_args: preset-jetson-h264
      inputs:
        - path: rtsp://xx:yy@192.168.0.96:554/cam/realmonitor?channel=1&subtype=1
          input_args: preset-rtsp-restream
          roles:
            - detect
            - audio
        - path: rtsp://xx:yy@192.168.0.96:554/cam/realmonitor?channel=1&subtype=0
          input_args: preset-rtsp-restream
          roles:
            - record
      output_args:
        record: preset-record-generic-audio-aac
    audio:
      enabled: false
      listen:
        - scream
        - speech
        - yell
        - skidding
        - tire_squeal
        - emergency_vehicle
        - gunshot
        - glass
        - crack
        - shatter
        - smash
        - breaking
    detect:
      width: 704
      height: 480
      fps: 10
    motion:
      threshold: 55
      contour_area: 20
      improve_contrast: 'true'
    zones:
      Neighbor_Exclude:
        coordinates: 0.344,0.25,1,0.26,0.998,0.992,0.006,0.995,0,0.311
        loitering_time: 0
    objects:
      filters:
        car:
          mask:
            - 0,0.414,0.285,0.4,0.317,1,0,1
            - 0.784,0.552,0.785,1,1,0.99,0.999,0.467
version: 0.14

[EtoTen]

Oh I just re-read your OP, you need to use sdkmanager to flash the latest JP5. Currently, Frigate will NOT work under JP6 without modification/updating of code.