SpatialPrompt: XR-Based Spatial Intent Expression as Executable Constraints for AI Generative 3D Design

TL;DR

SpatialPrompt is an XR system enabling users to sketch and voice-annotate spatial constraints for controllable 3D AI generation, supporting collaborative iterative design.

Contribution

It introduces a novel XR-based interface combining spatial sketches and voice prompts to control 3D generative models in a shared environment.

Findings

System is intuitive and supports shared understanding in collaborative creation.

Evaluation reveals the need for faster generation and clearer feedback.

Supports iterative refinement and synchronous co-creation.

Abstract

We present SpatialPrompt, an Extended Reality(XR) system that turns spatial sketches into executable constraints for controllable 3D generation. Users draw rough structures with a 3D pen and add voice prompts for semantic and stylistic intent. The system supports iterative refinement and synchronous co-creation in shared space with color-coded contributions. Implemented on Apple Vision Pro with Logitech Muse and Meshy, a heuristic evaluation suggests that the workflow is intuitive and supports shared understanding in collaborative creation, while revealing needs for faster generation and clearer feedback.