COARSE3D: Class-Prototypes for Contrastive Learning in Weakly-Supervised 3D Point Cloud Segmentation (BMVC 2022)

Rong Li ¹, Anh-Quan Cao ², Raoul de Charette ^2
1SCUT, China. ²Inria, France

If you find this work or code useful, please cite our paper and give this repo a star:

@inproceedings{coarse3d,
title={COARSE3D: Class-Prototypes for Contrastive Learning in Weakly-Supervised 3D Point Cloud Segmentation},
author={Rong Li, Anh-Quan Cao and Raoul de Charette},
booktitle={BMVC},
year={2022},
}

Teaser

Content

• Teaser
• Content
  • Environment
  • Preparing Dataset
    • SemanticKITTI
    • SemanticPOSS
    • nuScenes
  • Training
  • Trained Models
  • Acknowledgement
  • License

Environment

1. Create environment.

   conda create -n coarse3d python==3.8
   conda activate coarse3d
   

2. Install dependencies.

   conda install pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=10.2 -c pytorch
   pip install -r requirements.txt
   

3. Git clone and enter repo.

   git clone https://github.com/cv-rits/COARSE3D
   cd COARSE3D
   

Preparing Dataset

SemanticKITTI

1. Download SemanticKITTI dataset:

   • Download KITTI Odometry Benchmark Velodyne point clouds (80 GB)
   • Download SemanticKITTI label data (179 MB)

2. Enter subdir.

   cd tasks/prepare_data/
   

3. Set params in the gen_sem_weak_label_rand_grid.py, more details provided in the help comments of the parser.

4. Train

   python gen_sem_weak_label_rand_grid.py
   

   Then, the dataset strcuture should be like:

   └── semantic-kitti-coarse3d
       └── sequences
           ├── 00
           │   └── xxx # weakly-sup label
           ├── 01
           ...
           ...
   

SemanticPOSS

1. Download SemanticPOSS dataset:

   • LiDAR point cloud and label (2.2G)

2. Enter subdir.

   cd tasks/prepare_data/
   

3. Set params in the gen_sem_weak_label_rand_grid.py, more details provided in the help comments of the parser.

4. Generate weak label.

   python gen_sem_weak_label_rand_grid.py
   

   then, the data structure of weak label should be like:

   └── semantic-poss-coarse3d
       └── sequences
           ├── 00
           │   └── xxx # weakly-sup label
           ├── 01
           ...
           ...
   

nuScenes

1. Register and download nuScenes-lidarseg dataset.

2. Enter subdir.

   cd tasks/prepare_data/
   

3. Set params in the gen_nuscenes_weak_label_rand_grid.py, more details provided in the help comments of the parser.

4. Generate weak label.

   python gen_nuscenes_weak_label_rand_grid.py
   

   Then, the data structure should be like:

   └── nuscenes-coarse3d
      └── v1.0-trainval
           ├── 6d164dadab534fb087413b807bcf1de3_xxx.npy  # weakly-sup label
           ├── 6d0eb63329684744b3a6ffd4fbab830c_xxx.npy  # weakly-sup label
           ...
           ...
   

Training

1. Download ImageNet pretrained model.

2. Enter sub dir.

   cd tasks/weak_segmentation
   

3. Adapt config_dataset.yaml file (e.g. config_semantic_kitti.yaml), details are provided in the comments.

4. Adapt run.sh file. Replace the config_dataset.yaml with the dataset you are interested.

5. Train.

   bash run.sh
   

Trained Models

Dataset	Anno. (%)	Val mIoU (%)	Trained Model
SemanticKITTI	0.01	47.35	link
	0.1	57.57	link
SemanticPOSS	0.01	31.10	link
	0.1	43.00	link
nuScenes	0.01	42.90	link
	0.1	58.70	link

Acknowledgement

Rong Li was supported by the SMIL lab of South China University of Technolog, received support and advices from Prof. Mingkui Tan, Prof. Caixia Li and Zhuangwei Zhuang. Inria members were partly funded by French project SIGHT (ANR-20-CE23-0016). This work was performed using HPC resources from GENCI–IDRIS (Grant 2021-AD011012808 and 2022-AD011012808R1). We would also like to thank Ivan Lopes for the video voice-over.

License

COARSE3D is released under the Apache 2.0 license.