Isaac ROS 0.10.1 (DP)

README.md

@@ -30,6 +30,7 @@ The following are the benchmark performance results of the prepared pipelines in
     - [Docker](#docker)
   - [Quickstart](#quickstart)
   - [Next Steps](#next-steps)
+    - [Try More Examples](#try-more-examples)
     - [Try NITROS-Accelerated Graph with Argus](#try-nitros-accelerated-graph-with-argus)
     - [Use Different Models](#use-different-models)
     - [Customize your Dev Environment](#customize-your-dev-environment)
@@ -152,7 +153,9 @@ To simplify development, we strongly recommend leveraging the Isaac ROS Dev Dock
 
     > **Note:** The raw segmentation is also published to `/unet/raw_segmentation_mask`. However, the raw pixels correspond to the class labels and so the output is unsuitable for human visual inspection.
 ## Next Steps
-
+### Try More Examples
+To continue your exploration, check out the following suggested examples:
+- [Tutorial with Isaac Sim](./docs/tutorial-isaac-sim.md)
 ### Try NITROS-Accelerated Graph with Argus
 If you have an Argus-compatible camera, you can launch the NITROS-accelerated graph by following the [tutorial](docs/tutorial-nitros-graph.md).
 

docs/tutorial-isaac-sim.md

@@ -0,0 +1,50 @@
+# Tutorial with Isaac Sim
+> **Note: Isaac Sim 2022.1.0 published on 6/3/2022 does not support ROS2 Humble. Please follow one of the [workarounds](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/isaac-sim-sil-setup.md#isaac-sim-202210-workarounds) before continuing with the tutorial**
+1. Complete the [Quickstart section](../README.md#quickstart) in the main README till step 9.
+2. Launch the Docker container using the `run_dev.sh` script:
+    ```bash
+    cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
+      ./scripts/run_dev.sh
+    ```
+3. Inside the container, build and source the workspace:  
+    ```bash
+    cd /workspaces/isaac_ros-dev && \
+      colcon build --symlink-install && \
+      source install/setup.bash
+    ```
+4. Install and launch Isaac Sim following the steps in the [Isaac ROS Isaac Sim Setup Guide](https://github.com/NVIDIA-ISAAC-ROS/isaac_ros_common/blob/main/docs/isaac-sim-sil-setup.md)
+5. Open up the Isaac ROS Common USD scene (using the "content" window) located at:   
+   `omniverse://localhost/NVIDIA/Assets/Isaac/2022.1/Isaac/Samples/ROS2/Scenario/carter_warehouse_apriltags_worker.usd`.
+
+   And wait for it to load completely.
+   > **Note:** To use a different server, replace `localhost` with `<your_nucleus_server>`
+6. Go to the stage tab and select `/World/Carter_ROS`, then in properties tab -> Transform -> Translate -> X change `-3.0` to `0.0`.
+    <div align="center"><img src="../resources/Isaac_sim_set_carter.png" width="400px"/></div>
+</br>
+7.  Change the left camera topic name. Go to the stage tab and select `/World/Carter_ROS/ROS_Cameras/ros2_create_camera_left_rgb`, properties tab -> Compute Node -> Inputs -> topicName change `rgb_left` to `image`.
+    <div align="center"><img src="../resources/Isaac_sim_topic_rename.png" width="400px"/></div>
+</br>
+8. Press **Play** to start publishing data from the Isaac Sim application.
+    <div align="center"><img src="../resources/Isaac_sim_image_segmentation.png" width="800px"/></div>
+</br>
+9. Run the following launch files to start the inferencing:
+
+```bash
+    ros2 launch isaac_ros_unet isaac_ros_unet_triton.launch.py model_name:=peoplesemsegnet_shuffleseg model_repository_paths:=['/tmp/models'] input_binding_names:=['input_2:0'] output_binding_names:=['argmax_1'] network_output_type:='argmax'
+```
+</br>
+10. Visualize and validate the output of the package by launching `rqt_image_view` in another terminal:
+
+```bash
+cd ~/workspaces/isaac_ros-dev/src/isaac_ros_common && \
+      ./scripts/run_dev.sh
+```
+Then launch `rqt_image_view`:
+```bash
+    ros2 run rqt_image_view rqt_image_view
+```
+Then inside the `rqt_image_view` GUI, change the topic to `/unet/colored_segmentation_mask` to view a colorized segmentation mask. 
+
+<div align="center"><img alt="Coloured Segmentation Mask" src="../resources/Isaac_sim_peoplesemsegnet_shuffleseg_rqt.png" width="350" title="U-Net Shuffleseg result in rqt_image_view"/></div>
+
+**Note:** The raw segmentation is also published to `/unet/raw_segmentation_mask`. However, the raw pixels correspond to the class labels and so the output is unsuitable for human visual inspection.