Skip to content

Latest commit

 

History

History
98 lines (82 loc) · 2.94 KB

README.md

File metadata and controls

98 lines (82 loc) · 2.94 KB
Raw

3DFIRES: Few Image 3D REconstruction for Scenes with Hidden Surfaces

Linyi Jin1, Nilesh Kulkarni1, David F. Fouhey2
1University of Michigan 2New York University

teaser




3DFIRES reconstructs scene-level 3D from posed images, which works with as few as one view, reconstructs the complete geometry of unseen scenes, including hidden surfaces.





Install

conda env create -f environment.yml
conda activate 3dfires
# Download Pretrained Model
cd ckpts
wget https://www.dropbox.com/scl/fi/n5en8p4rqyudrj4kg8f3g/model_0299999.pth?rlkey=mva5crd8smkegkzgl41zh89tv
cd ..

In dropbox, we provide preprocessed file for testing, download and save to dataset/

wget -O dataset.tar "https://www.dropbox.com/scl/fi/hv4zn8s09vjsy0en0au0v/dataset.tar?rlkey=4910bmzoflnhpymfp1m1inhtj&dl=1"
tar -xvf dataset.tar

To get the full dataset for evaluation and training, you need to download the omnidata Taskonomy Medium dataset as tar files (no need to decompress since our code directly read from tar). We also need to create tarindex for faster IO.

conda install -c conda-forge aria2
pip install 'omnidata-tools'

omnitools.download rgb depth_zbuffer \
   --components taskonomy \
   --subset medium \
   --dest ./dataset/omnidata \
   --name YOUR_NAME --email YOUR_EMAIL --agree_all \
   --only_download True \
   --keep_compressed True

Create tarindexer for fast tar IO.

git clone git@github.com:devsnd/tarindexer.git
cd tarindexer
ROOT=../dataset/omnidata
tar=$ROOT/compressed
tarindex=$ROOT/tarindex
mkdir -p $tar
mkdir -p $tarindex
ls $tar | xargs -I{} python tarindexer.py -i $tar/{} $tarindex/{}

Demo

bash demo.sh

Evaluation

bash eval.sh

Training

bash train.sh

BibTeX

@article{jin20243dfires,
  author    = {Jin, Linyi and Kulkarni, Nilesh and Fouhey, David},
  title     = {3DFIRES: Few Image 3D REconstruction for Scenes with Hidden Surfaces},
  journal   = {CVPR},
  year      = {2024},
}