Create a live USB real time kernel with ROS 2 and benchmark tests

[awesomebytes]

Hello,

I apologize if this is not the best place to post it, but I was looking at the available documentation and resources and it came to my mind, that maybe the easiest (or second easiest if you have a raspberry pi 4) way to get people to try real time code could be with a live USB image preinstalled with the PREEMPT_RT kernel and ROS 2 with some demos & benchmarks.

I look forward to get more involved in the community. It was nice to meet some of you at ROSCON2022.

[awesomebytes]

To give myself an answer from the last Working group meeting (2022-Oct-25):
The point that focusing on one platform is a better investment of (limited) resources was raised. I do agree.
I may look into creating such an image and share my progress to not distract anyone else.

Meanwhile, I did try to source myself a raspberry pi 4, however, I was unable to purchase one at a reasonable price.
I opted for the most similar board I could find, a ROCK64 board: https://www.pine64.org/devices/single-board-computers/rock64/

4 x ARM Cortex A53 cores @ 1.5 GHz
ARM Mali 450 MP2 GPU
LPDDR3 RAM (up to 4GB)
Gigabit Ethernet
Micro SD Slot
eMMC module slot
SPI Flash 128Mbit
4K digital video out
2x USB 2.0 Host
1x USB 3.0 Host
PI-2 bus
PI-P5+ bus
IR R/X port
Real Time Clock (RTC) port
Power Over Ethernet (POE) (when using optional HAT module)
A/V jack
Power, Reset and Recovery buttons
3.5mm barrel power (5V 3A) port

About 45$ USD with 4GB RAM without the power supply included.

There's also the RockPro64 board, with 4 A53 cores at 1.4GHz, and 2 A72 cores at 1.8GHz for about 80$ USD, but I went for the closest to the Raspi 4.

[shuhaowu]

Having another board supported would be great! Here are some suggestions/notes:

• If you want to find a RPI4 for a reasonable price, you can follow https://rpilocator.com/. I think they may even have a RSS and twitter feed, if that is more convenient.
• In addition to the Pine64 board you mentioned above, there's also the ODroid N2+. For these boards, I would checkout Armbian as the base image.
• When buying boards, make sure they support arm64. Some boards only support armhf (such as the ODroid XU4) and this can cause issues these days as most software tend to support arm64 first. Not sure if ROS even has armhf builds.
• Keep in mind that most of these boards require the use of a patched kernel. Patching PREEMPT_RT on top may be another challenge as well. I've personally successfully done it for an ODROID XU4 in the past (not useful tho, because that thing is armhf only), but your mileage may vary depending on the board you get.
• Due to the supply chain issues over the last two years, my understanding is that the performance of most low-cost boards are roughly at the levels of a Raspberry Pi 4. The Rock64Pro and Odroid N2+ might be slightly faster, but it's not miles ahead.

[awesomebytes]

Little update:

I successfully created a Ubuntu 20.04 live cd with preemt_rt on kernel 5.15.

I documented how I followed the instructions to build the kernel here: https://gist.github.com/awesomebytes/8f9158c32a24930c16db6c529f7c2a50

And then documented here how I created the .iso using cubic here: https://gist.github.com/awesomebytes/19d864205557862156c60e51721f4f0b

The resulting image (uploading as I write this) is here: https://drive.google.com/drive/folders/1EiItg0NCjyovK-IDfgGpQJiL7ZTELl0F?usp=sharing (8.4GB, the original ubuntu-desktop is 3.9GB so there's room for improvement)

I tried it in Virtualbox and it worked:

And also on another laptop. I ran cyclictest briefly as to check if it worked (by cloning and building by hand, not included).

Maybe @shuhaowu you could give me some advice of what would be nice to bundle together exactly for quick tests in the image?
I guess benchmarking jitter/latency on different computers would be interesting too.

[shuhaowu]

You'll need to configure the image:

1. Use the performance governor and pin the CPU frequency
2. Turn off /proc/sys/kernel/sched_rt_runtime_us
3. Uninstall multipathd if installed

Then you can maybe try running cyclictest via this script: https://github.com/ros-realtime/ros-realtime-rpi4-image/blob/master/latency-analysis/run_latency_experiment.sh

You can then plot the results with: https://github.com/ros-realtime/ros-realtime-rpi4-image/blob/master/latency-analysis/plot_results.sh

These should be done on hardware (not virtualization).