From Air to Wear: Personalized 3D Digital Fashion with AR/VR Immersive 3D Sketching

TL;DR

This paper presents a user-friendly AR/VR system enabling non-experts to create personalized 3D digital garments from simple sketches, leveraging a novel diffusion model and a new dataset to improve accessibility and quality.

Contribution

It introduces a 3D sketch-driven garment generation framework with a shared latent space, adaptive learning, and a new dataset, making digital fashion design more accessible to ordinary users.

Findings

Outperforms existing methods in realism and usability.

Enables users without design experience to create high-quality digital clothing.

Demonstrates potential for democratizing virtual fashion creation.

Abstract

In the era of immersive consumer electronics, such as AR/VR headsets and smart devices, people increasingly seek ways to express their identity through virtual fashion. However, existing 3D garment design tools remain inaccessible to everyday users due to steep technical barriers and limited data. In this work, we introduce a 3D sketch-driven 3D garment generation framework that empowers ordinary users - even those without design experience - to create high-quality digital clothing through simple 3D sketches in AR/VR environments. By combining a conditional diffusion model, a sketch encoder trained in a shared latent space, and an adaptive curriculum learning strategy, our system interprets imprecise, free-hand input and produces realistic, personalized garments. To address the scarcity of training data, we also introduce KO3DClothes, a new dataset of paired 3D garments and user-created sketches. Extensive experiments and user studies confirm that our method significantly outperforms existing baselines in both fidelity and usability, demonstrating its promise for democratized fashion design on next-generation consumer platforms.