MeshXL is a family of generative pre-trained foundation models for 3D mesh generation. With the Neural Coordinate Field representation, the generation of unstructured 3D mesh data can be seaminglessly addressed by modern LLM methods.
The polygon mesh representation of 3D data exhibits great flexibility, fast rendering speed, and storage efficiency, which is widely preferred in various applications. However, given its unstructured graph representation, the direct generation of high-fidelity 3D meshes is challenging. Fortunately, with a pre-defined ordering strategy, 3D meshes can be represented as sequences, and the generation process can be seamlessly treated as an auto-regressive problem. In this paper, we validate the Neural Coordinate Field (NeurCF), an explicit coordinate representation with implicit neural embeddings, is a simple-yet-effective representation for large-scale sequential mesh modeling. After that, we present MeshXL, a family of generative pre-trained auto-regressive models, which addresses the process of 3D mesh generation with modern large language model approaches. Extensive experiments show that MeshXL is able to generate high-quality 3D meshes, and can also serve as foundation models for various down-stream applications.
- [2024/07/24] Upload the inference code and pre-trained weights.
- [2024/06/02] Upload paper and init project.
Environment Setting Up
You can build the environment using the provided script:
bash set_env.sh
Data
Work in progress...
Download Pre-Trained Weights
[Special Notes]: All the currently released weights are only used for unconditional 3D mesh generation.
We provide pre-trained weights for different sizes of models (i.e. 125m
, 350m
, and 1.3b
) on huggingface. Download the pre-trained weights from the links below to replace the pytorch_model.bin
files in the corresponding folders under the ./mesh-xl/
folder. The model details are shown below:
Model Size | #Layers | #Heads | GPU Hours | Download Link | ||
---|---|---|---|---|---|---|
125M | 12 | 12 | 768 | 3072 | 1944 | download link |
350M | 24 | 16 | 1024 | 4096 | 6000 | download link |
1.3B | 24 | 32 | 2048 | 8192 | 23232 | download link |
MeshXL Generative Pre-Training
Work in progress...
Generating Samples
![samples](/OpenMeshLab/MeshXL/raw/main/assets/images/objaverse-samples.png)
To generate 3D meshes with different sizes, feel free to use the following commands. By default, we generate samples with 8 GPUs and the top-k top-p sampling strategy for diverse samples.
bash scripts/sample-1.3b.sh
bash scripts/sample-350m.sh
bash scripts/sample-125m.sh
[Special Notes]: All the currently released weights are fine-tuned for unconditional 3D mesh generation on a specified category.
Want to generating shapes for a specified category? We have also uploaded the supervised fine-tuned checkpoints on chair
, table
, bench
, lamp
to huggingface too! Download the fine-tuned weights from the links below.
Model Size | Table | Chair | Lamp | Bench |
---|---|---|---|---|
125M | download link | download link | download link | download link |
350M | download link | download link | download link | download link |
1.3B | download link | download link | download link | download link |
After you have downloaded the corresponding checkpoints, please use the following command to generate samples.
export LLM_CONFIG='mesh-xl/mesh-xl-125m'
# the checkpoint mush align with the $LLM_CONFIG
export TEST_CKPT='./ckpts-meshxl-125m-sft/meshxl-125m-bench.pth'
accelerate launch \
--num_machines 1 \
--num_processes 8 \
--mixed_precision bf16 \
main.py \
--dataset dummy_dataset \
--n_max_triangles 800 \
--n_discrete_size 128 \
--llm mesh-xl/mesh-xl-125m \
--model mesh_xl \
--checkpoint_dir ./outputs \
--batchsize_per_gpu 2 \
--test_ckpt $TEST_CKPT \
--sample_rounds 100 \
--dataset_num_workers 0 \
--test_only
Want to see more results? Check our project page out here!
If you find our code or paper helps, please consider citing:
@misc{chen2024meshxl,
title={MeshXL: Neural Coordinate Field for Generative 3D Foundation Models},
author={Sijin Chen and Xin Chen and Anqi Pang and Xianfang Zeng and Wei Cheng and Yijun Fu and Fukun Yin and Yanru Wang and Zhibin Wang and Chi Zhang and Jingyi Yu and Gang Yu and Bin Fu and Tao Chen},
year={2024},
eprint={2405.20853},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
Thanks to these amazing works: ShapeNet, 3D-FUTURE, Objaverse, and Objaverse-XL.