Layout Generation Agents with Large Language Models

TL;DR

This paper introduces an agent-driven layout generation system using GPT-4V, which sequentially places objects in virtual spaces to efficiently create customizable 3D environments based on user instructions.

Contribution

It presents a novel approach leveraging multimodal large language models to generate virtual space layouts through agent manipulation, improving upon existing text-based methods.

Findings

High success rate in generating user-instructed virtual spaces

Effective manipulation of agents for sequential object placement

Identification of key elements improving behavior generation

Abstract

In recent years, there has been an increasing demand for customizable 3D virtual spaces. Due to the significant human effort required to create these virtual spaces, there is a need for efficiency in virtual space creation. While existing studies have proposed methods for automatically generating layouts such as floor plans and furniture arrangements, these methods only generate text indicating the layout structure based on user instructions, without utilizing the information obtained during the generation process. In this study, we propose an agent-driven layout generation system using the GPT-4V multimodal large language model and validate its effectiveness. Specifically, the language model manipulates agents to sequentially place objects in the virtual space, thus generating layouts that reflect user instructions. Experimental results confirm that our proposed method can generate virtual spaces reflecting user instructions with a high success rate. Additionally, we successfully identified elements contributing to the improvement in behavior generation performance through ablation study.