Sketch2BIM: A Multi-Agent Human-AI Collaborative Pipeline to Convert Hand-Drawn Floor Plans to 3D BIM

TL;DR

This paper presents a collaborative AI pipeline that transforms hand-drawn floor plans into accurate 3D BIM models by combining multimodal models, human feedback, and automated scripting, making BIM creation accessible and efficient.

Contribution

Introduces a novel multi-agent human-AI pipeline that converts sketches into 3D BIM models with high accuracy through iterative refinement and multimodal reasoning.

Findings

High reliability in capturing openings like doors and windows.

Wall detection accuracy improves to near 100% after feedback iterations.

Precision, recall, and F1 scores remain above 0.83 across categories.

Abstract

This study introduces a human-in-the-loop pipeline that converts unscaled, hand-drawn floor plan sketches into semantically consistent 3D BIM models. The workflow leverages multimodal large language models (MLLMs) within a multi-agent framework, combining perceptual extraction, human feedback, schema validation, and automated BIM scripting. Initially, sketches are iteratively refined into a structured JSON layout of walls, doors, and windows. Later, these layouts are transformed into executable scripts that generate 3D BIM models. Experiments on ten diverse floor plans demonstrate strong convergence: openings (doors, windows) are captured with high reliability in the initial pass, while wall detection begins around 83% and achieves near-perfect alignment after a few feedback iterations. Across all categories, precision, recall, and F1 scores remain above 0.83, and geometric errors (RMSE, MAE) progressively decrease to zero through feedback corrections. This study demonstrates how MLLM-driven multi-agent reasoning can make BIM creation accessible to both experts and non-experts using only freehand sketches.