Programmable-Room: Interactive Textured 3D Room Meshes Generation Empowered by Large Language Models
Jihyun Kim, Junho Park, Kyeongbo Kong, Suk-Ju Kang

TL;DR
Programmable-Room is a framework that uses large language models and visual programming to interactively generate and edit textured 3D room meshes from natural language instructions, enabling precise and flexible room design.
Contribution
It introduces a unified framework combining LLMs and visual programming for detailed 3D room mesh generation and editing from natural language, with novel modules including a diffusion-based panorama generator.
Findings
Demonstrates flexible and precise 3D room editing capabilities
Outperforms existing models in quality and usability
Utilizes a diffusion model with optimized training for panorama generation
Abstract
We present Programmable-Room, a framework which interactively generates and edits a 3D room mesh, given natural language instructions. For precise control of a room's each attribute, we decompose the challenging task into simpler steps such as creating plausible 3D coordinates for room meshes, generating panorama images for the texture, constructing 3D meshes by integrating the coordinates and panorama texture images, and arranging furniture. To support the various decomposed tasks with a unified framework, we incorporate visual programming (VP). VP is a method that utilizes a large language model (LLM) to write a Python-like program which is an ordered list of necessary modules for the various tasks given in natural language. We develop most of the modules. Especially, for the texture generating module, we utilize a pretrained large-scale diffusion model to generate panorama images…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
MethodsDiffusion · Long Short-Term Memory
