Skip to content

Latest commit

 

History

History
118 lines (85 loc) · 7.38 KB

GETTING_STARTED_PRETRAIN.md

File metadata and controls

118 lines (85 loc) · 7.38 KB
Raw

Getting Started for Point Cloud Pre-training

Please download and preprocess the point cloud datasets according to the dataset guidance

Download the labeled and pseudo-labeled data

How to run AD-PT

  • Take pre-training on small pseudo label set as an example:

      cd tools
  sh scripts/PRETRAIN/dist_train_ad-pt.sh ${NUM_GPUS} \
  --cfg_file ./cfgs/once_models/pretrain_models/once_ad-pt_pretrain_small.yaml

    or

      cd tools
  sh scripts/PRETRAIN/slurm_train_ad-pt.sh ${PARTITION} ${JOB_NAME} ${NUM_NODES} \ 
  --cfg_file ./cfgs/once_models/pretrain_models/once_ad-pt_pretrain_small.yaml

    Note you can choose small / medium / large pseudo set by changing the dataset config file (once_ad-pt_pretrain_small.yaml / once_ad-pt_pretrain_medium.yaml / once_ad-pt_pretrain_large.yaml)

  • Fine-tuning on downstream dataset:

      cd tools
  sh scripts/dist_train.sh ${NUM_GPUS} \
  --cfg_file ./cfgs/waymo_models/pv_rcnn_plusplus_resnet.yaml \
  --pretrained_model ${PRETRAINED_CHECKPOINT}

    or

      cd tools
  sh scripts/slurm_train.sh ${PARTITION} ${JOB_NAME} ${NUM_NODES} \
  --cfg_file ./cfgs/waymo_models/pv_rcnn_plusplus_resnet.yaml \
  --pretrained_model ${PRETRAINED_CHECKPOINT}

    ${PRETRAINED_CHECKPOINT} denotes the pre-trained checkpoints obtained using AD-PT method.

AD-PT pre-trained checkpoints

  • For rapid fine-tuning on downstream datasets, we also release the pre-trained checkpoint using our AD-PT

    Pre-training Method Pre-trained data Pre-trained model
    AD-PT ONCE PS-100K once-100K-ckpt
    AD-PT ONCE PS-500K once-500K-ckpt
    AD-PT ONCE PS-1M once-1M-ckpt

AD-PT Results:

We report the downstream fine-tuning results using our AD-PT pre-trained backbones.

Fine-tuning Results on Waymo:

Data amount Overall Vehicle Pedestrian Cyclist
SECOND (From scratch) 3% 52.00 / 37.70 58.11 / 57.44 51.34 / 27.38 46.57 / 28.28
SECOND (AD-PT) 3% 55.41 / 51.78 60.53 / 59.93 54.91 / 45.78 50.79 / 49.65
SECOND (From scratch) 20% 60.62 / 56.86 64.26 / 63.73 59.72 / 50.38 57.87 / 56.48
SECOND (AD-PT) 20% 61.26 / 57.69 64.54 / 64.00 60.25 / 51.21 59.00 / 57.86
CenterPoint (From scratch) 3% 59.00 / 56.29 57.12 / 56.57 58.66 / 52.44 61.24 / 59.89
CenterPoint (AD-PT) 3% 61.21 / 58.46 60.35 / 59.79 60.57 / 54.02 62.73 / 61.57
CenterPoint (From scratch) 20% 66.47 / 64.01 64.91 / 64.42 66.03 / 60.34 68.49 / 67.28
CenterPoint (AD-PT) 20% 67.17 / 64.65 65.33 / 64.83 67.16 / 61.20 69.39 / 68.25
PV-RCNN++ (From scratch) 3% 63.81 / 61.10 64.42 / 63.93 64.33 / 57.79 62.69 / 61.59
PV-RCNN++ (AD-PT) 3% 68.33 / 65.69 68.17 / 67.70 68.82 / 62.39 68.00 / 67.00
PV-RCNN++ (From scratch) 20% 69.97 / 67.58 69.18 / 68.75 70.88 / 65.21 69.84 / 68.77
PV-RCNN++ (AD-PT) 20% 71.55 / 69.23 70.62 / 70.19 72.36 / 66.82 71.69 / 70.70

Fine-tuning Results on nuScenes:

Data amount mAP NDS Car Truck CV. Bus Trailer Barrier Motorcycle Bicycle Pedestrian Cyclist
SECOND (From scratch) 5% 29.24 39.74 67.69 33.02 7.15 45.91 17.67 25.23 11.92 0.00 53.00 30.74
SECOND (AD-PT) 5% 37.69 47.95 74.89 41.82 12.05 54.77 28.92 34.41 23.63 3.19 63.61 39.54
SECOND (From scratch) 100% 50.59 62.29 - - - - - - - - - -
SECOND (AD-PT) 100% 52.23 63.04 83.12 52.86 15.24 68.58 37.54 59.48 46.01 20.44 78.96 60.05
CenterPoint (From scratch) 5% 42.68 50.41 77.82 43.61 10.65 44.01 18.71 52.95 36.26 16.76 37.62 54.52
CenterPoint (AD-PT) 5% 44.99 52.99 78.90 43.82 11.13 55.16 21.22 55.10 39.03 17.76 72.28 55.43
CenterPoint (From scratch) 100% 56.2 64.5 84.8 53.9 16.8 67.0 35.9 64.8 55.8 36.4 83.1 63.4
CenterPoint (AD-PT) 100% 57.17 65.48 84.86 54.37 16.09 67.354 36.06 64.31 58.50 40.58 83.53 66.05

Fine-tuning Results on KITTI:

Data amount mAP ( Mod.) Car (mod.) Pedestrian (Mod.) Cyclist (Mod.)
SECOND (From scratch) 20% 61.70 78.83 47.23 59.06
SECOND (AD-PT) 20% 65.95 80.70 49.67 67.50
SECOND (From scratch) 100% 66.70 80.78 52.61 66.71
SECOND (AD-PT) 100% 67.58 81.39 53.58 67.78
PV-RCNN (From scratch) 20% 66.71 82.52 53.33 64.28
PV-RCNN (AD-PT) 20% 69.43 82.75 57.59 67.96
PV-RCNN (From scratch) 100% 70.57 84.50 57.06 70.14
PV-RCNN (AD-PT) 100% 73.01 84.75 60.79 73.49