Skip to content

Commit

Permalink
Merge pull request #1436 from madeline-underwood/Himax
Browse files Browse the repository at this point in the history
Himax_AP Approved_Ready to go!
  • Loading branch information
pareenaverma authored Dec 13, 2024
2 parents fc5abce + deab4de commit 3ca1ec2
Show file tree
Hide file tree
Showing 6 changed files with 65 additions and 33 deletions.
22 changes: 10 additions & 12 deletions content/learning-paths/microcontrollers/yolo-on-himax/_index.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
---
title: Run a Computer Vision Model on a Himax Microcontroller
draft: true
cascade:
draft: true

minutes_to_complete: 90

who_is_this_for: This is an introduction topic explaining how to run a computer vision application on an embedded device from Himax. The example uses an off-the-shelf Himax WiseEye2 module which is based on Arm Cortex-M55 and Ethos-U55.
who_is_this_for: This is an introductory topic for developers who would like to learn about how to run a computer vision application on an embedded device from Himax.

learning_objectives:
- Run a you-only-look-once (YOLO) object detection model on the Himax device.
- Build the Himax Software Development Kit (SDK) and generate the firmware image file.
- Update the firmware on the Himax WiseEye2.
- Run a You-Only-Look-Once (YOLO) object detection model on a Himax WiseEye2 module.
- Build the Himax Software Development Kit (SDK) and generate a firmware image file.
- Update firmware on the Himax WiseEye2.
- Connect to and use Grove Vision AI module.

prerequisites:
- A [Seeed Grove Vision AI Module V2](https://www.seeedstudio.com/Grove-Vision-AI-Module-V2-p-5851.html) development board.
- An [OV5647-62 Camera Module](https://www.seeedstudio.com/OV5647-69-1-FOV-Camera-module-for-Raspberry-Pi-3B-4B-p-5484.html) and included FPC cable.
- An [OV5647-62 Camera Module](https://www.seeedstudio.com/OV5647-69-1-FOV-Camera-module-for-Raspberry-Pi-3B-4B-p-5484.html).
- A Flexible Printed Circuit (FPC) cable.
- A USB-C cable.
- An x86 Linux machine or a Mac running macOS with Apple Silicon.
- An x86 Linux machine, or a Mac running macOS.

author_primary: Chaodong Gong, Alex Su, Kieran Hejmadi

Expand All @@ -33,9 +33,7 @@ operatingsystems:
- Linux
- macOS

draft: true
cascade:
draft: true



### FIXED, DO NOT MODIFY
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Build the firmware
weight: 3
weight: 4

### FIXED, DO NOT MODIFY
layout: learningpathall
Expand All @@ -10,9 +10,9 @@ This section explains the process of generating a firmware image file.

## Clone the Himax GitHub project

Himax maintains a repository containing a few examples for the Seeed Grove Vision AI V2 board.
Himax maintains a repository containing a few examples that can be used with the Seeed Grove Vision AI V2 board.

It contains third-party software and scripts to build and flash the image with the object detection application. By recursively cloning the Himax examples repo, git will include the necessary sub-repositories that have been configured for the project.
It contains third-party software and scripts to build and flash the image with the object detection application. By recursively cloning the Himax examples repository, git includes the necessary subrepositories that have been configured for the project.

Clone the repository:

Expand All @@ -25,27 +25,27 @@ cd Seeed_Grove_Vision_AI_Module_V2

Use Make to compile the source code for object detection.

This takes up to 10 minutes depending on the number of CPU cores available on your host machine.
This can take up to 10 minutes depending on the number of CPU cores available on your host machine.

```bash
cd EPII_CM55M_APP_S
make
```

When the build is complete, you have an `.elf` file at `obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf`
When the build is complete, you will have an `.elf` file at `obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf`

## Generate the firmware image

The examples repository contains scripts to generate the image file.

Copy the `.elf` file to the `input_case1_secboot` directory.
Copy the `.elf` file to the `input_case1_secboot` directory:

```bash
cd ../we2_image_gen_local/
cp ../EPII_CM55M_APP_S/obj_epii_evb_icv30_bdv10/gnu_epii_evb_WLCSP65/EPII_CM55M_gnu_epii_evb_WLCSP65_s.elf input_case1_secboot/
```

Run the script your OS as shown below. This will create a file named `output.img` in the `output_case1_sec_wlcsp` directory.
Run the script on your OS as shown below. This creates a file named `output.img` in the `output_case1_sec_wlcsp` directory:


{{< tabpane code=true >}}
Expand All @@ -57,7 +57,7 @@ Run the script your OS as shown below. This will create a file named `output.img
{{< /tab >}}
{{< /tabpane >}}

The script output ends with the following output:
The script output ends with the following:

```output
Output image: output_case1_sec_wlcsp/output.img
Expand All @@ -66,4 +66,4 @@ Output image: output_case1_sec_wlcsp/output.img
IMAGE GEN DONE
```

You are ready to flash the image onto the Himax development board.
You are now ready to flash the image onto the Himax development board.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Set up the environment
weight: 2
weight: 3

### FIXED, DO NOT MODIFY
layout: learningpathall
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Flash firmware onto the microcontroller
weight: 4
weight: 5

### FIXED, DO NOT MODIFY
layout: learningpathall
Expand All @@ -10,7 +10,7 @@ Now that you have generated an image file on the local host machine, you are rea

## Install xmodem

`Xmodem` is a basic file transfer protocol which is easily installed using the Himax examples repository.
You can easily install a basic file transfer protocol called `Xmodem` using the Himax repository containing the examples.

Run the following command to install the dependency:

Expand All @@ -23,23 +23,30 @@ pip install -r xmodem/requirements.txt

It's time to get the board set up.

Insert the Flexible printed circuit (FPC) into the Grove Vision AI V2 module. Lift the dark grey latch on the connector as per the image below.
Insert the Flexible Printed Circuit (FPC) into the Grove Vision AI V2 module.

Lift the dark grey latch on the connector as shown in the image below.

![unlatched](./unlatched.jpg)

Slide the FPC connector in with the metal pins facing down and close the dark grey latch to fasten the connector.
* With the metal pins facing down, slide the FPC connector in.
* Close the dark grey latch to fasten the connector.

![latched](./latched.jpg)

Now you can connect the Groove Vision AI V2 Module to your computer via the USB-C cable.
Now you can connect the Groove Vision AI V2 Module to your computer using the USB-C cable.

{{% notice Note %}}
The development board may have two USB-C connectors. If you are running into issues connecting the board in the next step, make sure you are using the right one.
The development board might have two USB-C connectors. If you are running into issues connecting the board in the next step, make sure you are using the correct USB-C connector.
{{% /notice %}}

## Find the COM port

You'll need to provide the communication port (COM) which the board is connected to in order to flash the image. There are commands to list all COMs available on your machine. Once your board is connected through USB, it'll show up in this list. The COM identifier will start with **tty**, which may help you determine which one it is. You can run the command before and after plugging in the board if you are unsure.
To flash the image, you need to provide the communication port (COM) which the board is connected to.

On your machine, you can find commands that you can use to list all COMs available to use. Once your board is connected through USB, it will appear on this list of available COMs.

The COM identifier is prefixed with **tty**, so you can use this to help you identify which COM it is. You can also run the command before and after plugging in the board if you are unsure, and look for the change in the list.


{{< tabpane code=true >}}
Expand All @@ -53,14 +60,14 @@ ls /dev/tty.*


{{% notice Note %}}
If the port seems unavailable, try changing the permissions temporarily using the `chmod` command. Be sure to reset them afterwards, as this may pose a computer security vulnerability.
If the port appears to be unavailable, try changing the permissions temporarily using the `chmod` command. Be sure to reset the permissions again afterwards, as otherwise this can pose a computer security vulnerability.

```bash
chmod 0777 <COM port>
```
{{% /notice %}}

The full path to the port is needed in the next step, so be sure to save it.
You will require the full path to the port in the next step, so be sure to save it.

## Flash the firmware onto the module

Expand All @@ -82,7 +89,7 @@ After the firmware image flashing is completed, the message `Do you want to end

## Run the model

After the reset button is pressed, the board will start inference with the object detection automatically. Observe the output in the terminal to verify that the image is built correctly. If a person is in front of the camera, you should see the `person_score` value go over `100`.
After the reset button is pressed, the board starts inference with the object detection automatically. Observe the output in the terminal to verify that the image is built correctly. If a person is in front of the camera, you should see the `person_score` value exceed `100`.

```output
b'SENSORDPLIB_STATUS_XDMA_FRAME_READY 240'
Expand All @@ -97,4 +104,4 @@ b'person_score:112'
b'EVT event = 10'
```

This means the image works correctly on the device, and the end-to-end flow is complete.
This means the image works correctly on the device, and the end-to-end flow is complete.
24 changes: 24 additions & 0 deletions content/learning-paths/microcontrollers/yolo-on-himax/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
title: Overview
weight: 2

### FIXED, DO NOT MODIFY
layout: learningpathall
---

## What are the benefits of the Himax WiseEye2 Module?

* The Himax WiseEye2 implements the Arm-based Cortex M55 CPU and Ethos U55 NPU, and is one of the first "off-the-shelf" platforms of its type.

* The Himax WiseEye2 Module is an ultra-low power device that is energy-saving.

* It fully integrates with an existing Open Source AI framework (TFLite).

## What is a You-Only-Look-Once (YOLO) object detection model?

YOLO is a technology used in computer vision for identifying and locating objects in images and videos.

It is Open Source, fast, and has good detection accuracy.

To learn more about YOLO, see [Where to Start](https://docs.ultralytics.com/#where-to-start).

Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,9 @@ The images below are captured images from the models run in the toolkit.
### Objection detection
![object_detection](./object_detection.jpg)

The Frames Per Second (FPS) index represents the number of ML inferences the hardware can complete per second. A higher number indicates better performance. The colored bounding boxes represent the objects identified by YOLO. The name of the object is labelled in the top left-hand corner of the box, and the number in parentheses is the confidence level as a percentage. This example shows that it can identify 9.53 frames per second with a confidence level of 64% for the 'CPU' object.

### Face detection
![object_detection](./face_detection.jpg)

Similar to the previous example, the bounding boxes identify the areas in the image that contain faces and recognize the positions of different facial features. This image shows that YOLO has identified a face with 99% confidence. It has marked the mouth with a yellow line segment and used different colours to mark the eyebrows, eyes, and nose. Within the bounding box for the eyes, it has further identified the gaze direction vector.

0 comments on commit 3ca1ec2

Please sign in to comment.