LayoutGMN: Neural Graph Matching for Structural Layout Similarity

This repo provides the source code for our CVPR 2021 paper.

Overall, the repo consists of three parts:

1. Preparing Layout Graphs
2. Leveraging Graph Matching Networks (GMN)
3. Training GMN on the layout graphs

Preparing Layout Graphs

We used two kinds of layout data in our work: Floorplans and UI designs. Our code for layout graph data preparation is, in parts, borrowed from the work of Dipu et al.

In case you are looking for apn_dict (anchor-pos-neg) on floorplan data, here is the link to download it.

A few other pickle files that are needed for training on RPLAN data include: FP_box_info.pkl, FP_box_info_list.pkl and FP_data.p

Graph Matching Network-pytorch

The main machinery in our work is a Graph Matching Network that operates on the obtained layout graphs. We provide our own PyTorch implementation of Graph Matching Networks built on top of the Tensorflow Colab implementation by DeepMind Research.

Requirements

All the required modules are present in the requirements.txt file. Install all the requirements via

pip install -r requirements.txt

You should mainly care about these: Pytorch >=1.6, CUDA >= 9.1, networkx >= 2.3, torch-sparse==0.6.7 (pip install torch-sparse), torch-cluster==1.4.5 (pip install torch-cluster), torch-geometric==1.3.2 (pip install torch-geometric)

Citation

If you find our work useful in your research, consider citing:

@InProceedings{Patil_2021_CVPR,
    author    = {Patil, Akshay Gadi and Li, Manyi and Fisher, Matthew and Savva, Manolis and Zhang, Hao},
    title     = {LayoutGMN: Neural Graph Matching for Structural Layout Similarity},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2021},
    pages     = {11048-11057}
}