EchoLadder: Progressive AI-Assisted Design of Immersive VR Scenes

TL;DR

EchoLadder introduces an interactive VR design system using large vision-language models, enabling users to iteratively modify virtual scenes through verbal instructions, enhancing creativity and control in spatial design.

Contribution

It presents a novel human-AI collaboration pipeline that supports interactive, progressive scene modification in VR using large vision-language models, with toggle-controlled suggestions.

Findings

Effective pipeline components demonstrated in ablation study.

User study shows improved creativity with EchoLadder.

Provides insights into user strategies in AI-assisted design.

Abstract

Mixed reality platforms allow users to create virtual environments, yet novice users struggle with both ideation and execution in spatial design. While existing AI models can automatically generate scenes based on user prompts, the lack of interactive control limits users' ability to iteratively steer the output. In this paper, we present EchoLadder, a novel human-AI collaboration pipeline that leverages large vision-language model (LVLM) to support interactive scene modification in virtual reality. EchoLadder accepts users' verbal instructions at varied levels of abstraction and spatial specificity, generates concrete design suggestions throughout a progressive design process. The suggestions can be automatically applied, regenerated and retracted by users' toggle control.Our ablation study showed effectiveness of our pipeline components. Our user study found that, compared to baseline without showing suggestions, EchoLadder better supports user creativity in spatial design. It also contributes insights on users' progressive design strategies under AI assistance, providing design implications for future systems.