From 140a0c2ee36b3800d26e0f11f9e96605f99eaa14 Mon Sep 17 00:00:00 2001 From: Hemal Shah Date: Tue, 12 Jul 2022 12:56:38 -0700 Subject: [PATCH] Isaac ROS 0.10.1 (DP) --- README.md | 5 +- docs/tutorial-isaac-sim.md | 50 +++++++++++++++++++ resources/Isaac_sim_app_launcher.png | 3 -- resources/Isaac_sim_app_terminal.png | 3 -- resources/Isaac_sim_camera_view.png | 3 -- resources/Isaac_sim_common_scene.png | 3 -- resources/Isaac_sim_image_segmentation.png | 3 ++ ...aac_sim_peoplesemsegnet_shuffleseg_rqt.png | 3 ++ resources/Isaac_sim_set_carter.png | 3 ++ resources/Isaac_sim_topic_rename.png | 3 ++ resources/Segmentation_output.png | 3 -- resources/peoplesemsegnet_rviz2.png | 3 -- resources/peoplesemsegnet_segimage.png | 3 -- 13 files changed, 66 insertions(+), 22 deletions(-) create mode 100644 docs/tutorial-isaac-sim.md delete mode 100644 resources/Isaac_sim_app_launcher.png delete mode 100644 resources/Isaac_sim_app_terminal.png delete mode 100644 resources/Isaac_sim_camera_view.png delete mode 100644 resources/Isaac_sim_common_scene.png create mode 100644 resources/Isaac_sim_image_segmentation.png create mode 100644 resources/Isaac_sim_peoplesemsegnet_shuffleseg_rqt.png create mode 100644 resources/Isaac_sim_set_carter.png create mode 100644 resources/Isaac_sim_topic_rename.png delete mode 100644 resources/Segmentation_output.png delete mode 100644 resources/peoplesemsegnet_rviz2.png delete mode 100644 resources/peoplesemsegnet_segimage.png diff --git a/README.md b/README.md index 56ba37c..79c4415 100644 --- a/README.md +++ b/README.md @@ -30,6 +30,7 @@ The following are the benchmark performance results of the prepared pipelines in - [Docker](#docker) - [Quickstart](#quickstart) - [Next Steps](#next-steps) + - [Try More Examples](#try-more-examples) - [Try NITROS-Accelerated Graph with Argus](#try-nitros-accelerated-graph-with-argus) - [Use Different Models](#use-different-models) - [Customize your Dev Environment](#customize-your-dev-environment) @@ -152,7 +153,9 @@ To simplify development, we strongly recommend leveraging the Isaac ROS Dev Dock > **Note:** The raw segmentation is also published to `/unet/raw_segmentation_mask`. However, the raw pixels correspond to the class labels and so the output is unsuitable for human visual inspection. ## Next Steps - +### Try More Examples +To continue your exploration, check out the following suggested examples: +- [Tutorial with Isaac Sim](./docs/tutorial-isaac-sim.md) ### Try NITROS-Accelerated Graph with Argus If you have an Argus-compatible camera, you can launch the NITROS-accelerated graph by following the [tutorial](docs/tutorial-nitros-graph.md). diff --git a/docs/tutorial-isaac-sim.md b/docs/tutorial-isaac-sim.md new file mode 100644 index 0000000..2e608e2 --- /dev/null +++ b/docs/tutorial-isaac-sim.md @@ -0,0 +1,50 @@ +# Tutorial with Isaac Sim +> **Note: Isaac Sim 2022.1.0 published on 6/3/2022 does not support ROS2 Humble. Please follow one of the [workarounds](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/isaac-sim-sil-setup.md#isaac-sim-202210-workarounds) before continuing with the tutorial** +1. Complete the [Quickstart section](../README.md#quickstart) in the main README till step 9. +2. Launch the Docker container using the `run_dev.sh` script: + ```bash + cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \ + ./scripts/run_dev.sh + ``` +3. Inside the container, build and source the workspace: + ```bash + cd /workspaces/isaac_ros-dev && \ + colcon build --symlink-install && \ + source install/setup.bash + ``` +4. Install and launch Isaac Sim following the steps in the [Isaac ROS Isaac Sim Setup Guide](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/isaac-sim-sil-setup.md) +5. Open up the Isaac ROS Common USD scene (using the "content" window) located at: + `omniverse://localhost/NVIDIA/Assets/Isaac/2022.1/Isaac/Samples/ROS2/Scenario/carter_warehouse_apriltags_worker.usd`. + + And wait for it to load completely. + > **Note:** To use a different server, replace `localhost` with `` +6. Go to the stage tab and select `/World/Carter_ROS`, then in properties tab -> Transform -> Translate -> X change `-3.0` to `0.0`. +
+
+7. Change the left camera topic name. Go to the stage tab and select `/World/Carter_ROS/ROS_Cameras/ros2_create_camera_left_rgb`, properties tab -> Compute Node -> Inputs -> topicName change `rgb_left` to `image`. +
+
+8. Press **Play** to start publishing data from the Isaac Sim application. +
+
+9. Run the following launch files to start the inferencing: + +```bash + ros2 launch isaac_ros_unet isaac_ros_unet_triton.launch.py model_name:=peoplesemsegnet_shuffleseg model_repository_paths:=['/tmp/models'] input_binding_names:=['input_2:0'] output_binding_names:=['argmax_1'] network_output_type:='argmax' +``` +
+10. Visualize and validate the output of the package by launching `rqt_image_view` in another terminal: + +```bash +cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \ + ./scripts/run_dev.sh +``` +Then launch `rqt_image_view`: +```bash + ros2 run rqt_image_view rqt_image_view +``` +Then inside the `rqt_image_view` GUI, change the topic to `/unet/colored_segmentation_mask` to view a colorized segmentation mask. + +
Coloured Segmentation Mask
+ +**Note:** The raw segmentation is also published to `/unet/raw_segmentation_mask`. However, the raw pixels correspond to the class labels and so the output is unsuitable for human visual inspection. diff --git a/resources/Isaac_sim_app_launcher.png b/resources/Isaac_sim_app_launcher.png deleted file mode 100644 index a17e73e..0000000 --- a/resources/Isaac_sim_app_launcher.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:cfbf5fbcee70cb8ccecd5846573b568a8046930d933c7e533881aecd64ab3e9a -size 67843 diff --git a/resources/Isaac_sim_app_terminal.png b/resources/Isaac_sim_app_terminal.png deleted file mode 100644 index fcd8aa0..0000000 --- a/resources/Isaac_sim_app_terminal.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:448d244a3eae344c4bb69a328d48e1a938d30b1325fb307d18fb042bf516c250 -size 57060 diff --git a/resources/Isaac_sim_camera_view.png b/resources/Isaac_sim_camera_view.png deleted file mode 100644 index 5aa9649..0000000 --- a/resources/Isaac_sim_camera_view.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:16161d3143fa859efff3855119d7c4bdda4edc268b202237c0a5283644ebd809 -size 1266329 diff --git a/resources/Isaac_sim_common_scene.png b/resources/Isaac_sim_common_scene.png deleted file mode 100644 index 31eab6d..0000000 --- a/resources/Isaac_sim_common_scene.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:e26a90fbb2fba1435c3b0d38cc9131d3591f5cbc3cdb34d6741b8820476d24f2 -size 1285567 diff --git a/resources/Isaac_sim_image_segmentation.png b/resources/Isaac_sim_image_segmentation.png new file mode 100644 index 0000000..e6b2576 --- /dev/null +++ b/resources/Isaac_sim_image_segmentation.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:b3c9d5a87d20764a1cf399727e411d27b41a2e629e8fdbfb2ceb8deb5a4efb5a +size 885214 diff --git a/resources/Isaac_sim_peoplesemsegnet_shuffleseg_rqt.png b/resources/Isaac_sim_peoplesemsegnet_shuffleseg_rqt.png new file mode 100644 index 0000000..cd5caf9 --- /dev/null +++ b/resources/Isaac_sim_peoplesemsegnet_shuffleseg_rqt.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:f094cab4011b52f5c8a3a2866c37a974c30f8ddf828c18eab804967c2e8ef1e9 +size 26762 diff --git a/resources/Isaac_sim_set_carter.png b/resources/Isaac_sim_set_carter.png new file mode 100644 index 0000000..ad70c9c --- /dev/null +++ b/resources/Isaac_sim_set_carter.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:4d4d7a5ea61e5b85b4eebf62b3b0daba2ffebe68ce894d20050f6c1cbd9a5f07 +size 61329 diff --git a/resources/Isaac_sim_topic_rename.png b/resources/Isaac_sim_topic_rename.png new file mode 100644 index 0000000..fe4c8bf --- /dev/null +++ b/resources/Isaac_sim_topic_rename.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:55d5ad1cd5a04e370f99984dfac6ef4d95d6c8f33a80722ed924a28bd27bac72 +size 72222 diff --git a/resources/Segmentation_output.png b/resources/Segmentation_output.png deleted file mode 100644 index 7d19edf..0000000 --- a/resources/Segmentation_output.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:697fa04501b3ce11cc9d08f5775515e3d2b93ccfde13c08aeda33a366c1c8242 -size 31116 diff --git a/resources/peoplesemsegnet_rviz2.png b/resources/peoplesemsegnet_rviz2.png deleted file mode 100644 index 6526627..0000000 --- a/resources/peoplesemsegnet_rviz2.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:0c0d662b0e42d26c872c0d73c2f2bd7033b8f342f4df968b5294e2d383933947 -size 260071 diff --git a/resources/peoplesemsegnet_segimage.png b/resources/peoplesemsegnet_segimage.png deleted file mode 100644 index 4e66e1a..0000000 --- a/resources/peoplesemsegnet_segimage.png +++ /dev/null @@ -1,3 +0,0 @@ -version https://git-lfs.github.com/spec/v1 -oid sha256:bd9bd0ead5b7a492deab5aa1044d884589f197abe14318886200f967406a5334 -size 8371