This application demonstrates use of the Edge Impulse C++ Inferencing Libarary with the Golioth Firmware SDK on a Nordic Thingy91 with Zephyr (nRF Connect SDK). It uses a model based on the continuous gestures dataset. Gesture classification results are streamed to Golioth's timeseries database, while raw accelerometer sensor readings are delivered to an object storage bucket where they can later be imported into Edge Impulse for model training. Data routing is configured using Golioth Pipelines.
You will need Golioth and Edge Impulse accounts to use this application. After signing up, go through the following steps.
- Setup your Zephyr workspace.
west init -m https://github.com/golioth/example-edge-impulse.git --mf west-ncs.yml
west update
- Create a project on Golioth.
- Create a project on Edge Impulse.
In order to deliver classification results and raw accelerometer data to their respective destinations, two pipelines must be created in the target Golioth project.
Classification Results LightDB Stream
This pipeline accepts CBOR data on the /class path and converts it to JSON
before delivering it to LightDB Stream, Golioth's timeseries database.
Click here to create this pipeline in your project.
This pipeline accepts the binary accelerometer readings and delivers them to an Amazon S3 object storage bucket, where they can later be imported into Edge Impulse.
[!NOTE] Make sure to create secrets for the referenced
AWS_ACCESS_KEYandAWS_SECRET_KEY, and update thenameandregionto match your bucket.
Click here to create this pipeline in your project.
To perform gesture classification on-device, a model and inferencing library will need to be generated. Follow the instructions in the continuous motion recognition tutorial to generate and download a C++ library.
After downloading, use the following commands to extract components and include them in the application directory.
mkdir ei-generated
unzip -q <path-to-generated>.zip -d ei-generated/
mv ei-generated/edge-impulse-sdk/ .
mv ei-generated/model-parameters/ .
mv ei-generated/tflite-model/ .
After acquiring all necessary components, the firmware application can be built using the following command.
west build -p -b thingy91_nrf9160_ns app/
After successful build, the device can be programmed with the following command.
west flash
The firmware requires credentials in order to communicate with Golioth. Credentials can be provided at runtime using the Zephyr shell via a serial connection to the device.
uart:~$ settings set golioth/psk-id <my-psk-id@my-project>
uart:~$ settings set golioth/psk <my-psk>
On button press, the application will start sampling data from the Thingy91's accelerometer, then use the trained model to classify the motion as a gesture (e.g. wave, snake, etc.). The results of the classification, as well as the raw sampled data will then be streamed to Golioth.
The following is an example of classification data that can be viewed in the Golioth console.
{
"idle": 0.28515625,
"snake": 0.16796875,
"updown": 0.21875,
"wave": 0.328125
}Accelerometer data is uploaded as an array of floats representing X-Y-Z position data. Each sampling will result in a new object being created in the specified S3 bucket.
Pull requests and issues can be submitted at https://github.com/golioth/example-edge-impulse. Code changes will be mirrored to https://github.com/edgeimpulse/example-golioth.
This project will only accept contributions using the Developer’s Certificate of Origin 1.1 located at https://developercertificate.org (“DCO”). The DCO is a legally binding statement asserting that you are the creator of your contribution, or that you otherwise have the authority to distribute the contribution, and that you are intentionally making the contribution available under the license associated with the Project ("License").
You can agree to the DCO in your contribution by using a “Signed-off-by” line at the end of your commit message. You should only submit a contribution if you are willing to agree to the DCO terms. If you are willing, just add a line to the end of every git commit message:
Signed-off-by: Jane Smith <jane.smith@email.com>