Monocular Depth Estimation Toolbox and Benchmark

 

Introduction

MMDepth is an open source monocular depth estimation toolbox based on PyTorch. The main branch works with PyTorch 1.6+.

Note that this repo is not a part of the OpenMMLab project. It is built on top of MMSegmentation. If there is any infringement please contact ruijiezhu@mail.ustc.edu.cn.

🎉 Introducing mmdepth v1.0.0 🎉

Since this is the first version of this repo, there are inevitably many imperfections. The code library inherits most of the functions and advantages of MMSegmentation, and adds or expands some components. We welcome you to develop and maintain this repo together!

Major features

• Unified Benchmark

  We provide a unified toolbox and benchmark for training or testing models on multiple depth datasets.

• Modular Design

  Following OpenMMLab series, we decompose the monocular depth estimation models into different components. As a result, it is easy to construct a customized monocular depth estimation model by combining different modules.

User guides

Please refer to get_started.md for installation and dataset_prepare.md for dataset preparation.

Please refer to train.md for model training and inference.md for model inference.

For detailed guides for development, please see tutorials from MMSegmentation.

Projects

Here are some implementations of SOTA models and solutions built on MMDepth, which are supported and maintained by community users. These projects demonstrate the best practices based on MMDepth for research and product development. We welcome and appreciate all the contributions to these projects. Also, we appreciate all contributions to improve MMDepth framework.

	ScaleDepth: Decomposing Metric Depth Estimation into Scale Prediction and Relative Depth Estimation Ruijie Zhu, Chuxin Wang, Ziyang Song, Li Liu, Tianzhu Zhang, Yongdong Zhang Arxiv, 2024 [Paper] [Webpage] [Code]
	BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation Zhenyu Li, Xuyang Wang, Xianming Liu, and Junjun Jiang IEEE Transactions on Image Processing, 2024 [Paper] [Code]

Benchmark and model zoo

Results and models are available in the model zoo.

Overview

Supported datasets	Supported backbones	Supported head	Supported loss
NYU-Depth V2 KITTI SUN RGB-D iBims-1 Benchmark DIODE HyperSim Virtual KITTI 2 DDAD DIML	ResNet(CVPR'2016) ResNeXt (CVPR'2017) HRNet (CVPR'2019) ResNeSt (ArXiv'2020) MobileNetV2 (CVPR'2018) MobileNetV3 (ICCV'2019) Vision Transformer (ICLR'2021) Swin Transformer (ICCV'2021) Twins (NeurIPS'2021) BEiT (ICLR'2022) ConvNeXt (CVPR'2022) MAE (CVPR'2022) PoolFormer (CVPR'2022) SegNeXt (NeurIPS'2022) CLIP (ICML'2021)	ANN_Head APC_Head ASPP_Head CC_Head DA_Head DDR_Head DM_Head DNL_Head DPT_HEAD EMA_Head ENC_Head FCN_Head FPN_Head GC_Head LightHam_Head ISA_Head Knet_Head LRASPP_Head mask2former_Head maskformer_Head NL_Head OCR_Head PID_Head point_Head PSA_Head PSP_Head SAN_Head segformer_Head segmenter_mask_Head SepASPP_Head SepFCN_Head SETRMLAHead_Head SETRUP_Head STDC_Head Uper_Head VPDDepth_Head ScaleDepth_Head BinsFormer_Head	boundary_loss cross_entropy_loss dice_loss focal_loss huasdorff_distance_loss kldiv_loss lovasz_loss ohem_cross_entropy_loss silog_loss tversky_loss

Acknowledgement

MMDepth is an open source project maintained by the author alone (at least for now). The original intention of building this project is to provide a standardized toolbox and benchmark for monocular depth estimation. We thank the contributers of MMSegmentation for providing a great template for this project. We also thank the authors of Monocular-Depth-Estimation-Toolbox and ZoeDepth, whose code this project borrowed.

Citation

If you find this project useful in your research, please consider cite:

@misc{mmdepth2024,
    title={{mmdepth}: Monocular Depth Estimation Toolbox and Benchmark},
    author={Ruijie Zhu},
    howpublished = {\url{https://github.com/RuijieZhu94/mmdepth}},
    year={2024}
}

License

This project is released under the Apache 2.0 license.