TC-Depth

Project Page | arXiv | Workshop

Official Repository for

Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimation

Daoyi Gao, Hanzhi Chen, Patrick Ruhkamp, Nassir Navab, Benjamin Busam - 3DV, 2021.

Spatial-Temporal Attention through Self-Supervised Geometric Guidance

Daoyi Gao, Hanzhi Chen, Patrick Ruhkamp, Nassir Navab, Benjamin Busam - ICCV Workshop on Self-supervised Learning for Next-Generation Industry-level Autonomous Driving, 2021.

Bold: equal contribution

🤓 TL;DR

• Current SOTA in self-supervised monocular depth estimation achievies highly accurate depth predictions, but suffer from inconsistencies across temporal frames
• Our novel Spatial-Temporal Attention mechanism with Geometric Guidance improves consistency while maintaining accuracy
• The Temporal Consistency Metric (TCM) is a quantitative measure to evaluate the consistency between temporal predictions in 3D

🎇 News

• Pretrained weight available (04.11.2022)
• Release training code (02.10.2022)
• Evaluation code for TCM available (02.12.2021)

🖊 Results

Qualitative Results

Spatial-Temporal Attention

Temporal Consistency Metric (TCM)

💽 Resource

GT for TCM

3 Frames Track | 5 Frames Track | 7 Frames Track

Pretrained Weights

KITTI DRNC-26

📄 Citation

If you find our work useful, please consider citing the following papers:

@inproceedings{ruhkamp2021attention,
    title = {Attention meets Geometry: Geometry Guided Spatial-Temporal Attention for Consistent Self-Supervised Monocular Depth Estimation},
    author = {Patrick Ruhkamp and
              Daoyi Geo and
              Hanzhi Chen and
              Nassir Navab and
              Benjamin Busam},
    booktitle = {IEEE International Conference on 3D Vision (3DV)},
    year = {2021},
    month = {December}
}

@article{monodepth2,
  title     = {Digging into Self-Supervised Monocular Depth Prediction},
  author    = {Cl{\'{e}}ment Godard and
               Oisin {Mac Aodha} and
               Michael Firman and
               Gabriel J. Brostow},
  booktitle = {The International Conference on Computer Vision (ICCV)},
  month = {October},
year = {2019}
}

❤ Acknowledgement

Our implementation is based on MonoDepth2 and follows their code structure. Thanks for their great contribution :)