Replies: 3 comments 34 replies
-
|
I don't think it will work unless it's compatible with tensorflow lite. |
Beta Was this translation helpful? Give feedback.
-
|
According to the documentation, it supports TensorFlow, TensorFlow Lite, ONNX, Keras and Pytorch. |
Beta Was this translation helpful? Give feedback.
-
|
Raspberry pi foundation chose Hailo 8 chip for their Pi AI hat. install base should rise a bunch. |
Beta Was this translation helpful? Give feedback.
-
|
I would really be curious to try out an Hailo adapter if Frigate supported that. |
Beta Was this translation helpful? Give feedback.
-
|
Refreshing the subject as Hailo modules are now part of the offering as the Pi5 AI kits. |
Beta Was this translation helpful? Give feedback.
-
|
+1 would love to see hailo supported! |
Beta Was this translation helpful? Give feedback.
-
|
+1 from me.. these Hailo-8L modules the AI Hat comes with have seemingly 3.25x the TOPS than the Coral modules have at a much lower Watt/TOP efficiency number (see https://hailo.ai/files/hailo-8l-m-2-et-product-brief-en/). Would really be neat if Frigate would support those. |
Beta Was this translation helpful? Give feedback.
-
|
Ok thanks you I have now got the detector loading in frigate 😁. But I am still having problems, now with the python file I wrote to handle the frames from frigate, well not frames I guess as I found out frigate outputs tensors so I'm now having to work out how to handle them. I found a network file from hailo model zoo the I think is a good fit, it the yolov5s. Anyway the problem I'm having is this I start frigate it loads and initialise fine but I don't think I'm handling the data I get right as I get errors: frigate-h8l | 2024-07-02 04:46:18.841995297 [HailoRT] [error] CHECK failed - Memory size of vstream yolov5s_spp/input_layer1 does not match the frame count! (Expected 4915200, got 1228800)
frigate-h8l | 2024-07-02 04:46:18.842054445 [HailoRT] [error] CHECK_SUCCESS failed with status=HAILO_INVALID_ARGUMENT(2)
frigate-h8l | 2024-07-02 04:46:18.844410269 Process detector:hailo8l:
frigate-h8l | 2024-07-02 04:46:18.844816712 [2024-07-02 04:46:18] frigate.detectors.plugins.hailo8l ERROR : HailoRTException during inference: Invalid argument. See hailort.log for more information
frigate-h8l | 2024-07-02 04:46:18.845214248 Traceback (most recent call last):
frigate-h8l | 2024-07-02 04:46:18.845218303 File "/usr/lib/python3.9/multiprocessing/process.py", line 315, in _bootstrap
frigate-h8l | 2024-07-02 04:46:18.845219229 self.run()
frigate-h8l | 2024-07-02 04:46:18.845220396 File "/usr/lib/python3.9/multiprocessing/process.py", line 108, in run
frigate-h8l | 2024-07-02 04:46:18.845221377 self._target(*self._args, **self._kwargs)
frigate-h8l | 2024-07-02 04:46:18.845223562 File "/opt/frigate/frigate/object_detection.py", line 127, in run_detector
frigate-h8l | 2024-07-02 04:46:18.845230470 outputs[connection_id]["np"][:] = detections[:]
frigate-h8l | 2024-07-02 04:46:18.845233025 ValueError: could not broadcast input array from shape (0,) into shape (20,6)And this is the Hailort log: [2024-07-02 05:06:04.599] [482] [HailoRT] [info] [device.cpp:46] [Device] OS Version: Linux 6.6.31+rpt-rpi-2712 #1 SMP PREEMPT Debian 1:6.6.31-1+rpt1 (2024-05-29) aarch64
[2024-07-02 05:06:04.599] [482] [HailoRT] [info] [control.cpp:100] [control__parse_identify_results] firmware_version is: 4.17.0
[2024-07-02 05:06:04.599] [482] [HailoRT] [info] [vdevice.cpp:566] [create] VDevice Infos: 0000:01:00.0
[2024-07-02 05:06:04.702] [482] [HailoRT] [info] [internal_buffer_manager.cpp:208] [print_execution_results] Planned internal buffer memory: CMA memory 0, user memory 2543104. memory to edge layer usage factor is 0.8264559
[2024-07-02 05:06:04.702] [482] [HailoRT] [info] [internal_buffer_manager.cpp:216] [print_execution_results] Default Internal buffer planner executed successfully
[2024-07-02 05:06:04.725] [482] [HailoRT] [info] [device_internal.cpp:58] [configure] Configuring HEF took 29.300103 milliseconds
[2024-07-02 05:06:04.725] [482] [HailoRT] [info] [vdevice.cpp:665] [configure] Configuring HEF on VDevice took 29.593786 milliseconds
[2024-07-02 05:06:04.725] [482] [HailoRT] [info] [hef.cpp:1802] [get_network_group_and_network_name] No name was given. Addressing all networks of default network_group: yolov5s_spp
[2024-07-02 05:06:04.725] [482] [HailoRT] [info] [hef.cpp:1802] [get_network_group_and_network_name] No name was given. Addressing all networks of default network_group: yolov5s_spp
[2024-07-02 05:06:04.726] [482] [HailoRT] [info] [core_op.cpp:160] [activate] Activating yolov5s_spp took 0.394313 milliseconds. Note that the function is asynchronous and thus the network is not fully activated yet.
[2024-07-02 05:06:04.726] [482] [HailoRT] [info] [core_op.cpp:160] [activate] Activating yolov5s_spp took 0.415313 milliseconds. Note that the function is asynchronous and thus the network is not fully activated yet.
[2024-07-02 05:06:04.726] [482] [HailoRT] [info] [edge_elements.cpp:55] [create] Created (HwWriteEl1yolov5s_spp/input_layer1 | hw_frame_size: 1228800)
[2024-07-02 05:06:04.727] [482] [HailoRT] [info] [queue_elements.cpp:255] [create] Created (PushQEl1yolov5s_spp/input_layer1 | timeout: 10s)
[2024-07-02 05:06:04.729] [482] [HailoRT] [info] [filter_elements.cpp:110] [create] Created (PreInferEl1yolov5s_spp/input_layer1 | Quantization - src_type: FLOAT32, dst_type UINT8, limvals_min: 0, limvals_max: 255 | Reorder - src_order: NHWC, src_shape: (640, 640, 3), dst_order: NHCW, dst_shape: (640, 640, 3))
[2024-07-02 05:06:04.729] [482] [HailoRT] [info] [vstream.cpp:754] [InputVStreamImpl] Creating yolov5s_spp/input_layer1...
[2024-07-02 05:06:04.729] [482] [HailoRT] [info] [vstream_builder.cpp:101] [create_inputs] Input pipeline 'yolov5s_spp/input_layer1': (PreInferEl1yolov5s_spp/input_layer1 | Quantization - src_type: FLOAT32, dst_type UINT8, limvals_min: 0, limvals_max: 255 | Reorder - src_order: NHWC, src_shape: (640, 640, 3), dst_order: NHCW, dst_shape: (640, 640, 3)) >> (PushQEl1yolov5s_spp/input_layer1 | timeout: 10s) >> (HwWriteEl1yolov5s_spp/input_layer1 | hw_frame_size: 1228800) >> HW
[2024-07-02 05:06:04.729] [482] [HailoRT] [info] [multi_io_elements.cpp:156] [create] Created (NmsPPMuxEl0YOLOv5-Post-Process | Op YOLOV5, Name: YOLOv5-Post-Process, Score threshold: 0.001, IoU threshold: 0.60, Classes: 80, Cross classes: false, Max bboxes per class: 80, Image height: 640, Image width: 640)
[2024-07-02 05:06:04.729] [482] [HailoRT] [info] [edge_elements.cpp:293] [create] Created (HwReadEl3yolov5s_spp/conv55_106 | hw_frame_size: 1632000)
[2024-07-02 05:06:04.729] [482] [HailoRT] [info] [queue_elements.cpp:613] [create] Created (PullQEl_nms3yolov5s_spp/conv55_106 | timeout: 10s)
[2024-07-02 05:06:04.729] [482] [HailoRT] [info] [edge_elements.cpp:293] [create] Created (HwReadEl0yolov5s_spp/conv63_106 | hw_frame_size: 408000)
[2024-07-02 05:06:04.729] [482] [HailoRT] [info] [queue_elements.cpp:613] [create] Created (PullQEl_nms0yolov5s_spp/conv63_106 | timeout: 10s)
[2024-07-02 05:06:04.729] [482] [HailoRT] [info] [edge_elements.cpp:293] [create] Created (HwReadEl1yolov5s_spp/conv70_106 | hw_frame_size: 122400)
[2024-07-02 05:06:04.730] [482] [HailoRT] [info] [queue_elements.cpp:613] [create] Created (PullQEl_nms1yolov5s_spp/conv70_106 | timeout: 10s)
[2024-07-02 05:06:04.730] [482] [HailoRT] [info] [queue_elements.cpp:756] [create] Created (UserBufQEl_post_infer0yolov5s_spp/yolov5_nms_postprocess | timeout: 10s)
[2024-07-02 05:06:04.730] [482] [HailoRT] [info] [vstream.cpp:1107] [OutputVStreamImpl] Creating yolov5s_spp/yolov5_nms_postprocess...
[2024-07-02 05:06:04.730] [482] [HailoRT] [info] [vstream_builder.cpp:621] [create_output_post_process_nms] Output pipeline 'yolov5s_spp/yolov5_nms_postprocess': HW >> (HwReadEl3yolov5s_spp/conv55_106 | hw_frame_size: 1632000) >> (PullQEl_nms3yolov5s_spp/conv55_106) >> (HwReadEl0yolov5s_spp/conv63_106 | hw_frame_size: 408000) >> (PullQEl_nms0yolov5s_spp/conv63_106) >> (HwReadEl1yolov5s_spp/conv70_106 | hw_frame_size: 122400) >> (PullQEl_nms1yolov5s_spp/conv70_106) >> (NmsPPMuxEl0YOLOv5-Post-Process | Op YOLOV5, Name: YOLOv5-Post-Process, Score threshold: 0.001, IoU threshold: 0.60, Classes: 80, Cross classes: false, Max bboxes per class: 80, Image height: 640, Image width: 640) >> (UserBufQEl_post_infer0yolov5s_spp/yolov5_nms_postprocess | timeout: 10s)
[2024-07-02 05:06:04.730] [482] [HailoRT] [error] [inference_pipeline.cpp:274] [verify_memory_view_size] CHECK failed - Memory size of vstream yolov5s_spp/input_layer1 does not match the frame count! (Expected 4915200, got 1228800)
[2024-07-02 05:06:04.730] [482] [HailoRT] [error] [inference_pipeline.cpp:201] [infer] CHECK_SUCCESS failed with status=HAILO_INVALID_ARGUMENT(2) |
Beta Was this translation helpful? Give feedback.
-
|
Yolo5 can't be integrated into Frigate due to the same license issues as yolov8. Technically you need a license to even do what you already did. |
Beta Was this translation helpful? Give feedback.
-
|
oops ok I have removed it. what models from Hailo model Zoo would I be ok to use then? |
Beta Was this translation helpful? Give feedback.
-
|
I would suggest getting started with ssd_mobilenet v1 or v2 |
Beta Was this translation helpful? Give feedback.
-
|
Ok, it is running without crashing. now I've got to get it to show what it finds on a video stream. :-) |
Beta Was this translation helpful? Give feedback.
-
@blakeblackshear I have been waiting to use your awesome module but am wanting to know it it is compatible with https://hailo.ai/
Beta Was this translation helpful? Give feedback.
All reactions