This repository is developed in Fish-IoT project
https://www.tequ.fi/en/project-bank/fish-iot/
This guide is for using Tensorflow 1 (tfjs-node-gpu) in Node-RED using Jetson Nano or Xavier NX device and run object detection on images. Might work on Xavier AGX also, but I didnt have one to test. At the moment of writing this, tfjs-node(-gpu) directly depends on libtensorflow version 1.15.0, so downgrading CUDA on Jetson Xavier NX is necessary to make things work. If you are using Jetson Nano, you can install Jetpack 4.3 from official NVIDIA Jetpack 4.3 image and start from list item number 9.
UPDATE 31.8.2020
Newest tfjs-node-gpu versions (tested with 3.8.0 and 3.9.0) work with newer libtensorflow versions. Check this guide if you want to use Tensorflow 2.
https://github.com/Lapland-UAS-Tequ/tequ-jetson-nodered-tensorflow/
After running all commands you should have following versions of the components
| Software | Version |
|---|---|
| Jetpack | 4.5.1 or 4.3 |
| CUDA | 10.0.326 |
| cuDNN | 7.6.3.28 |
| libtensorflow | 1.15.0 |
| node-red | 2.0.5 |
| tfjs-node-gpu | 1.4.0 |
https://developer.nvidia.com/embedded/learn/getting-started-jetson
sudo apt update && sudo apt upgrade
sudo apt purge cuda-tools-10-2 libcudnn8 cuda-documentation-10-2 cuda-samples-10-2 nvidia-l4t-graphics-demos ubuntu-wallpapers-bionic libreoffice* chromium-browser* thunderbird fonts-noto-cjk
sudo apt autoremove
sudo reboot
cd /home/
cd /<your user name>/
mkdir cuda_files
cd /home/cuda_files
wget https://jetson-nodered-files.s3.eu.cloud-object-storage.appdomain.cloud/cuda-repo-l4t-10-0-local-10.0.326_1.0-1_arm64.deb
wget https://jetson-nodered-files.s3.eu.cloud-object-storage.appdomain.cloud/libcudnn7_7.6.3.28-1+cuda10.0_arm64.deb
sudo dpkg -i cuda-repo-l4t-8-0-local_8.0.34-1_arm64.deb
sudo apt update
sudo apt search cuda
sudo apt install cuda-toolkit-10.0
sudo apt install cuda-samples-10.0
export PATH=/usr/local/cuda-10.0/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-10.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
sudo apt install ./libcudnn7_7.6.3.28-1+cuda10.0_arm64.deb
sudo apt-get install python3-pip
sudo -H pip3 install -U jetson-stats
jtop
bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)
https://docs.nvidia.com/deeplearning/frameworks/install-tf-jetson-platform/index.html
sudo apt-get update
sudo apt-get install libhdf5-serial-dev hdf5-tools libhdf5-dev zlib1g-dev zip libjpeg8-dev liblapack-dev libblas-dev gfortran
sudo pip3 install -U pip testresources setuptools==49.6.0
sudo pip3 install -U numpy==1.19.4 future==0.18.2 mock==3.0.5 h5py==2.10.0 keras_preprocessing==1.1.1 keras_applications==1.0.8 gast==0.2.2 futures protobuf pybind11
wget https://jetson-nodered-files.s3.eu.cloud-object-storage.appdomain.cloud/tensorflow_gpu-1.15.0+nv20.1-cp36-cp36m-linux_aarch64.whl
sudo pip3 install tensorflow-gpu/tensorflow_gpu-1.15.0+nv20.1-cp36-cp36m-linux_aarch64.whl
python3
import tensorflow
cd ~/.node-red
npm install @tensorflow/tfjs-node-gpu@1.4.0
npm install @cloud-annotations/models-node-gpu
Installation will finish with errors. Ignore errors and continue.
cd ~/.node-red/node_modules/@tensorflow/tfjs-node-gpu/deps
wget https://jetson-nodered-files.s3.eu.cloud-object-storage.appdomain.cloud/libtensorflow-gpu-linux-arm64-1.15.0.tar.gz
tar xzvf libtensorflow-gpu-linux-arm64-1.15.0.tar.gz
sudo npm install --global node-pre-gyp
npm run build-addon-from-source
test
cd ~/.node-red
node
var tf = require('@tensorflow/tfjs-node-gpu')
https://www.npmjs.com/package/canvas
Install dependencies first
sudo apt-get install build-essential libcairo2-dev libpango1.0-dev libjpeg-dev libgif-dev librsvg2-dev
npm install canvas
Start Node-RED
node-red-start
Go to:
https://github.com/Lapland-UAS-Tequ/tequ-api-client/
Copy and import 'example-ai-detect-v2.json' to your Node-RED.
You should see something like this in Node-RED log after flow is deployed, if everything regarding to Tensorflow went well:
Configure model folder
Inject image to flow and start detecting objects.
First inference is slow and it takes something like ~5-30 seconds. After that it should run smoothly.
If you need to build your own model, you can follow this guide:
https://github.com/Lapland-UAS-Tequ/tequ-tf1-ca-training-pipeline
GUI is disabled
sudo service gdm stop
sudo systemctl set-default multi-user.target
Inference speeds for MJPEG stream from Raspberry PI4 with HQ-camera
Streaming is started with command
raspivid -v -n -b 25000000 -qp 10 -md 2 -w 1920 -h 1080 -fps 25 -cd MJPEG -n -rot 180 -t 0 -o tcp://127.0.0.1:50001
*md (mode) and w and h parameters can vary.
Raspivid MJPEG stream is parsed in Node-RED at RPI4 and rerouted to Jetson via Websocket.
NVIDIA Jetson Xavier NX
| Resolution | FPS | Frame size |
|---|---|---|
| 320 x 240 | 15 | ~35 kB |
| 1280 x 720 | 10 | ~115 kB |
| 1920 x 1080 | 8 | ~122 kB |
| 4000 x 600 | 8 | ~112 kB |
| 2028 x 1520 | 5 | ~121 kB |
| 4056 x 1520 | 3 | ~137 kB |
NVIDIA Jetson Nano
| Resolution | FPS | Frame size |
|---|---|---|
| 320 x 240 | 8 | ~35 kB |
| 1280 x 720 | 6 | ~115 kB |
| 1920 x 1080 | 5 | ~122 kB |
| 4000 x 600 | 4 | ~112 kB |
| 2028 x 1520 | 3 | ~121 kB |
| 4056 x 1520 | 2 | ~137 kB |