Skip to content

Commit

Permalink
added viz
Browse files Browse the repository at this point in the history
  • Loading branch information
gianscarpe committed Mar 4, 2024
1 parent 96b510e commit 97797e5
Show file tree
Hide file tree
Showing 7 changed files with 22 additions and 11 deletions.
29 changes: 20 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,28 @@

![Method](https://raw.githubusercontent.com/IIT-PAVIS/DiffAssemble/release/page/DiffAssemble.png)

Reassembly tasks play a fundamental role in many fields and multiple approaches exist to solve specific reassembly problems. In this context, we posit that a general unified model can effectively address them all, irrespective of the input data type (images, 3D, etc.). We introduce DiffAssemble, a Graph Neural Network (GNN)-based architecture that learns to solve reassembly tasks using a diffusion model formulation. Our method treats the elements of a set, whether pieces of 2D patch or 3D object fragments, as nodes of a spatial graph. Training is performed by introducing noise into the position and rotation of the elements and iteratively denoising them to reconstruct the coherent initial pose. DiffAssemble achieves state-of-the-art (SOTA) results in most 2D and 3D reassembly tasks and is the first learning-based approach that solves 2D puzzles for both rotation and translation. Furthermore, we highlight its remarkable reduction in run-time, performing 11 times faster than the quickest optimization-based method for puzzle solving. We will release the code upon paper acceptance.
- [Webpage](https://iit-pavis.github.io/DiffAssemble/)
- [Code](https://github.com/IIT-PAVIS/DiffAssemble)
- [Arxiv](soon)
- [Arxiv](https://arxiv.org/abs/2402.19302")

# Method
![Method](https://raw.githubusercontent.com/IIT-PAVIS/DiffAssemble/release/page/DiffAssemble.png)
![Architecture](https://raw.githubusercontent.com/IIT-PAVIS/DiffAssemble/release/page/architecture.jpg)
Following the Diffusion Probabilistic
Models formulations, we model a Markov chain where we inject noise into the pieces’ position and orientation. At timestep t = 0, the
pieces are in their correct position, and at timestep t = T , they are in a random position with random orientation. At each timestep t, our
attention-based GNN takes as input a graph where each node contains an equivariant feature that describes a particular piece and its position
and orientation. The network then predicts a less noisy version of the piece’s position and orientation.

# Visuals

## 2D
![2d](https://raw.githubusercontent.com/IIT-PAVIS/DiffAssemble/release/page/viz.gif)

## 3D

![3d](https://raw.githubusercontent.com/IIT-PAVIS/DiffAssemble/release/page/bottle_new2_0001-0350.gif)

# Datasets

## Puzzles
Expand Down Expand Up @@ -46,12 +58,11 @@ and orientation. The network then predicts a less noisy version of the piece’s

# Cite us!
```
@misc{giuliari2023positional,
title={Positional Diffusion: Ordering Unordered Sets with Diffusion Probabilistic Models},
author={Francesco Giuliari and Gianluca Scarpellini and Stuart James and Yiming Wang and Alessio {Del Bue}},
year={2023},
eprint={2303.11120},
archivePrefix={arXiv},
primaryClass={cs.CV}
@InProceedings{scarpellini2024diffassemble,
author = {Gianluca Scarpellini and Stefano Fiorini and Francesco Giuliari and Pietro Morerio and Alessio Del Bue},
title = {DiffAssemble: A Unified Graph-Diffusion Model for 2D and 3D Reassembly},
booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
month = {June},
year = {2024},
}
```
4 changes: 2 additions & 2 deletions _config.yaml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
remote_theme: pages-themes/cayman@v0.2.0
plugins:
- jekyll-remote-theme
title: DiffAssemble
description: Official repository for DiffAssemble-A Unified Graph-Diffusion Model for 2D and 3D Reassembly accepted at CVPR2024
title: DiffAssemble-A Unified Graph-Diffusion Model for 2D and 3D Reassembly
description: Accepted at CVPR2024
Binary file added page/architecture.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added page/bottle_new2_0001-0350.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added page/celeba_14_1.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added page/celeba_14_2.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added page/wikiart_6_1.gif
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 97797e5

Please sign in to comment.