LLM4DESIGN: An Automated Multi-Modal System for Architectural and Environmental Design

TL;DR

LLM4DESIGN is an automated multi-modal system that generates coherent architectural and environmental design proposals using site data, multi-agent creativity, retrieval augmentation, and visual language models, advancing urban renewal design.

Contribution

It introduces a novel integrated system combining multi-agent, retrieval-augmented generation, and visual language models for automated architectural and environmental design.

Findings

Demonstrates superior performance in urban renewal design tasks.

Confirms the system's ability to produce realistic and creative design proposals.

Provides the first cross-modal design scheme dataset for architecture and urban design.

Abstract

This study introduces LLM4DESIGN, a highly automated system for generating architectural and environmental design proposals. LLM4DESIGN, relying solely on site conditions and design requirements, employs Multi-Agent systems to foster creativity, Retrieval Augmented Generation (RAG) to ground designs in realism, and Visual Language Models (VLM) to synchronize all information. This system resulting in coherent, multi-illustrated, and multi-textual design schemes. The system meets the dual needs of narrative storytelling and objective drawing presentation in generating architectural and environmental design proposals. Extensive comparative and ablation experiments confirm the innovativeness of LLM4DESIGN's narrative and the grounded applicability of its plans, demonstrating its superior performance in the field of urban renewal design. Lastly, we have created the first cross-modal design scheme dataset covering architecture, landscape, interior, and urban design, providing rich resources for future research.