SHOWMe: Benchmarking Object-agnostic Hand-Object 3D Reconstruction (ICCVW 2023) (ACVR Oral)

[Paper] [Project Page]

Anilkumar Swamy, Vincent Lerory, Philippe Weinzaepfel, Fabien Baradel, Salma Galaaoui, Romain Brégier, Matthieu Armando, Jean-Sebastien Franco, Grégory Rogez
ICCVW 2023

This repository contains the link for downloading and code for visualizing SHOWMe Hand-Object dataset.

RGBs Mosaic

Dataset Information

Citation

@inproceedings{showme,
  title={{SHOWMe: Benchmarking Object-agnostic Hand-Object 3D Reconstruction}},
  author={{Swamy, Anilkumar and Leroy, Vincent and Weinzaepfel, Philippe and Baradel, Fabien and Galaaoui, Salma and Brégier, Romain and Armando, Matthieu and Franco, Jean-Sebastien and Rogez, Grégory}},
  booktitle={{ICCVW}},
  year={2023}
}

News

• Top baseline 3D Recnostructions and Camera Poses for quantitative and qualitative comparison
• Dataset Release
• Visualization Scripts Release

Dataset Download

Link

3D recnostruction models and Camera Poses

To make qualitative and quantitative comparison easier, we release best baseline 3D reconstruction models(mesh files) and the corresponding camera poses, researchers can download these and use for comparision instead of running the whole pipeline again

showme_baseline_recons_camposes

Scripts

• To print all sequence ids
• To visualize RGB frame
• To visualize RGBD frame
• To visualize GT meshes (hand-object and object)
• To visualize projected meshes
• To visualize mano mesh
• To visualize pixel-aligned depth point clouds
• To render semantic maps (render mano hand and object mesh)

Running Scripts

Install

Our code is running using python3.7 and requires the following packages:

• pytorch-1.7.1+cu110
• pytorch3d-0.3.0
• PIL
• numpy
• trimesh
• matplotlib

Display all sequence ids

python scripts/dataset_info.py --datadir <dataset_directory_path> 

Visualize

python scripts/vis_showme.py --vis_type <visualization_type> --datadir <dataset_directory_path> --depth_datadir <depth_dataset_directory_path> --seq_id <sequence_id> --frm_no <frame_number>

Arguments help:

• <vis_type>: visualization type argument ( rgb or rgbd or depth or ho_mesh or obj_mesh or mano_mesh or proj_verts or pix_algnd_depth )

Render (renders mano hand and object)

python scripts/render.py --datadir <dataset_directory_path> --outdir <output_dir_tosave_rendered_images> --seq_id <sequence_id>

License

The code is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 license.

A summary of the CC BY-NC-SA 4.0 license is located here: https://creativecommons.org/licenses/by-nc-sa/4.0/

The CC BY-NC-SA 4.0 license is located here: https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode