- Ubuntu 16.04/18.04/20.04 with compatible ROS version
- Python >= 3.8
- gcc & g++ >= 5.4
- OpenCV 3 or 4
- Nvidia GPU (with compatible cuda toolkit and cuDNN) if want to run online segmentation
- Anaconda for configuring python dependencies
First create and navigate to your catkin workspace
cd <your-working-directory>
mkdir <your-ros-ws>/src && cd <your-ros-ws>
Then, initialize the workspace and configure it. (Remember to replace by your ros version)
catkin init
catkin config --extend /opt/ros/<your-ros-version> --merge-devel
catkin config --cmake-args -DCMAKE_CXX_STANDARD=14 -DCMAKE_BUILD_TYPE=Release
Download this repository to your ROS workspace src/
folder with submodules via:
cd src
git clone --recursive https://github.com/hmz-15/Interactive-Scene-Reconstruction.git
Then add dependencies specified by .rosinstall using wstool. For your conivenience, we provide a script to automatically setup the wstool configuration. Please make sure you run the script at the root of your ros workspace.
cd <your-ros-ws>
sh src/Interactive-Scene-Reconstruction/wstool_setup_https.sh
Noted that switch to wstool_setup_ssh.sh
if you are using ssh instead of https.
We assume using conda virtual environment to configure python dependencies. This requires Anaconda to be installed and initialized as prerequisite. We create a conda env with python3.7
and install the dependencies:
conda create --name robot-scene-recon python=3.7 -y
conda activate robot-scene-recon
pip install pip --upgrade
# detectron2 dependencies
pip install torch torchvision
python -m pip install detectron2 -f \
https://dl.fbaipublicfiles.com/detectron2/wheels/cu102/torch1.10/index.html
# other dependencies
cd src/Interactive-Scene-Reconstruction
pip install -r requirements.txt
Note that you may need to adjust the version of torch, torchvision and detectron2 based on your cuda/cuDNN version.
You can deactivate the conda env using:
conda deactivate
We first build the python package containing panoptic segmentation server.
conda activate robot-scene-recon
cd mapping/rp_server
make dev
Then we build the ros packages with catkin build
.
cd <your-ros-ws>
catkin build panoptic_mapping_pipeline map_proc scene_builder gazebo_simulation -j2
source devel/setup.bash
Alternatively, you may want to build packages for each individule module:
# Build packages for panoptic mapping
catkin build panoptic_mapping_pipeline -j2
# Build packages for cad replacement
catkin build map_proc
# Build packages for interactive scene reconstruction & gazebo demo
catkin build scene_builder gazebo_simulation
Please replace bash
by zsh
if zsh
is your default shell.