*/}\n
\n Recent advancements in diffusion models for 2D and 3D content creation have sparked a surge of interest in generating 4D content. However, the scarcity of 3D scene datasets constrains current methodologies to primarily object-centric generation. To overcome this limitation, we present Comp4D, a novel framework for Compositional 4D Generation. Unlike conventional methods that generate a singular 4D representation of the entire scene, Comp4D innovatively constructs each 4D object within the scene separately. Utilizing Large Language Models (LLMs), the framework begins by decomposing an input text prompt into distinct entities and maps out their trajectories. It then constructs the compositional 4D scene by accurately positioning these objects along their designated paths. To refine the scene, our method employs a compositional score distillation technique guided by the pre-defined trajectories, utilizing pre-trained diffusion models across text-to-image, text-to-video, and text-to-3D domains. Extensive experiments demonstrate our outstanding 4D content creation capability compared to prior arts, showcasing superior visual quality, motion fidelity, and enhanced object interactions.\n
\n\n TL;DR: We introduce Comp4D, compositional 4D scene synthesis from text input. Compared with previous object-centric 4D generation pipelines, our Compositional 4D Generation (Comp4D) framework integrates GPT-4 to decompose the scene and design proper trajectories, resulting in larger-scale movements and more realistic object interactions.\n
\n {/*\n Previous work concentrate on object-centric 4D objects. In comparison, our work extends the boundaries to the demanding task of constructing compositional 4D scenes. We integrate GPT-4 to decompose the scene and design proper trajectories, resulting in larger-scale movements and more realistic object interactions.\n
\n\n Here's an overview of our proposed Comp4D method. Given an input text, first, we use LLM for scene decomposition to obtain multiple individual 3D objects. Subsequently, we adopt LLM to design the object trajectories which guide the displacements of objects in the optimization of compositional 4D scene. Thanks to the compositional 4D representation implemented with 3D Gaussians, in each iteration of the compositional score distillation process, we are able to switch between object-centric rendering and trajectory-guided rendering flexibly.\n
\n\n Here's the overall pipeline for object trajectory generation. First, a scene description is provided by a human user as a prompt to an LLM which yields relative object scale information required for rendering. Subsequently, the language model is prompted with this information along with environmental constraints and tasked to return a function that takes timestep as an input and returns the corresponding object's 3D position governed by kinematic equations. After the collection of a set of positions, collision checking is performed to clip the trajectory where the first collision occurs. Optionally, premature collisions can be mitigated by re-querying the LLM for an improved function.\n
\n\n Our method is evaluated against per-frame baselines and concurrent method Consistent4D. We also provide image-to-4D results by using Stable Video Diffusion to generate a video from an image.\n
\n
\n If you want to cite our work, please kindly use:\n
\n \n {`@article{comp4d,\n title={Comp4D: LLM-Guided Compositional 4D Scene Generation},\n author={},\n journal={arXiv preprint},\n year={2023}\n}\\}`}\n \n \n \n