patient-monitoring

Jack Schofield, Null Atwood, Travis Hudson, Rohan B Ballapragada, Zachary Taylor, Jet Ittihrit, Aditya Raj

ROS2 packages for the patient-monitoring robot. For the patient-monitoring Android application, go to pmr-app-test.

Example GIF

System Diagram

Requirements

Hardware:

TurtleBot3 Waffle
Intel® RealSense™ Depth Camera

Note: The patient-monitoring nodes were tested using an Intel RealSense™ D405, but should function with any camera compatible with the Intel® RealSense™ ROS2 wrapper. Please inform us of any issues that arise using different camera models.

Software:

Ubuntu 20.04 (link)
ROS2 Foxy (link)
Intel® RealSense™ SDK 2.0 (link)
Intel® RealSense™ ROS2 wrapper (link)

Installation

Before continuing, please install Ubuntu 20.04 on the Turtlebot3's onboard computer. If you plan on using a remote PC, it will require Ubuntu 20.04 (and a ROS2 foxy install) as well. This guide will not cover setting up a remote PC on ROS2.

Step 1: Install ROS2 Foxy and TurtleBot3 Packages

The TurtleBot3 Quick Start Guide (Section 3) is a good resource for setting up our ROS2 environment. Please refer to it (and make sure you are following the steps for "Foxy").

In our usage, we replaced the onboard Raspberry Pi that ships with the Turtlebot3 with an Intel® NUC to allow for real-time image processing onboard the robot. If you are planning to do something similar, please see the annotated notes below:

Section 3. 1. PC Setup can be applied to the onboard PC, as we will require a working ROS2 installation to run nodes directly on the robot.
- Section 3. 1. 6. Network Configuration can be skipped, although we recommend completing it. Even if you want your image processing nodes to run directly on the robot, it is still useful to have a remote PC to see system diagnostics.
Section 3. 2. SBC Setup can be entirely skipped if you are not using a Raspberry Pi, except for:
- Section 3. 2. 6. Configure the Raspberry Pi for those using a remote PC setup (the Raspberry Pi correspondsROS_DISTRO=<YOUR_SYSTEM_ROS_DISTRO> # set your ROS_DISTRO: humble, galactic, foxy source /opt/ros/$ROS_DISTRO/setup.bash cd ~/ros2_ws . install/local_setup.bash to the onboard PC, in our case the NUC).
- Section 3. 2. 7. NEW LDS-02 Configuration if you are using an LDS-02 (2022+ TurtleBot3 models). ROS2 packages default to the older LDS-01.
Section 3. 3. OpenCR Setup can be followed, but we recommend attempting to bring-up the robot first and seeing if the current firmware on the OpenCR board is functional.
Section 3. 5. 1. Bringup TurtleBot3 demonstrates run basic TurtleBot3 programs and test basic operation.
- You do not need to follow Step 1 of 3. 5. 1. if you are running onboard the robot.
- Step 2 of 3. 5. 1. recommends running export TURTLEBOT3_MODEL=burger. You will have to run this command every terminal emulator instance. Instead, we recommend export TURTLEBOT3_MODEL=burger >> ~/.bashrc to save this environmental variable.
Section 3. 6. 1. Teleoperation is a good way to test functionality of your robot past simply running the turtlebot3_bringup node. If your robot does not respond to teleoperation commands, we've documented some common issues we ran into below:
- Ensure the 12V power and the USB to the OpenCR board is plugged in. If only the USB is connected, ROS2 can successfully communicate with the board, but the board cannot send movements to the wheels. ROS2 will not display any error messages if this occurs.
- Ensure the USB port the OpenCR is plugged into has appropriate privileges (/dev/ttyACM0 by default). chmod 777 /dev/ttyACM0. This is also a common issue with the LDS-01/02 (/dev/ttyUSB0 by default)

Step 2: Install the Intel® RealSense™ SDK 2.0 and ROS2 Wrapper for for Intel® RealSense™ Devices

The realsense-ros repository's README contains installation instructions for the Intel® RealSense™ SDK 2.0 and ROS2 Wrapper.

Step 3: Install the patient-monitoring ROS2 package

Create a ROS2 workspace (this should have been completed in Step 2).

mkdir -p ~/ros2_ws/
cd ~/ros2_ws/

Clone the patient-monitoring package and copy the /src directory to your ROS2 workspace.

git clone https://github.com/schofija/patient-monitoring
cp -r patient-monitoring/src/ ~/ros2_ws/

Install dependencies

sudo apt-get install python3-rosdep -y
sudo rosdep init # "sudo rosdep init --include-eol-distros" for Eloquent and earlier
rosdep update # "sudo rosdep update --include-eol-distros" for Eloquent and earlier
rosdep install -i --from-path src --rosdistro $ROS_DISTRO --skip-keys=librealsense2 -y

Build patient_monitoring and my_robot_interfaces package

colcon build --packages-select patient_monitoring && colcon build --packages-select my_robot_interfaces

Source environment (do this in a different terminal than the one you ran colcon build in to avoid weird errors)

cd ~/ros2_was
. install/setup.bash

Usage

Turtlebot3 and RealSense™ Nodes

The turtlebot3_bringup and realsense2_camera_node nodes must be running for the patient_monitoring nodes to function. Option 1: Run the robot_launch launchfile

cd ~/ros2_ws/src/launch
ros2 launch robot_launch.py

Option 2: Manually launch nodes

ros2 launch turtlebot3_bringup robot.launch.py
ros2 launch realsense2_camera rs_launch.py depth_module.profile:=1280x720x30 align_depth.enable:=true

Please ensure your environment is sourced to avoid 'package not found' errors.