VR-Doh: Hands-on 3D Modeling in Virtual Reality

TL;DR

VR-Doh is an open-source VR system that enables intuitive, real-time 3D modeling and editing of elastoplastic objects through natural hand interactions, enhancing accessibility and immersive creative workflows.

Contribution

It introduces a novel real-time simulation and rendering pipeline tailored for VR-based 3D modeling of deformable objects, combining customized MPM and Gaussian Splatting techniques.

Findings

Users found VR-Doh intuitive and immersive for creative tasks.

System achieves real-time performance with smooth interactions.

Supports diverse modeling applications like food, characters, and structures.

Abstract

We introduce VR-Doh, an open-source, hands-on 3D modeling system that enables intuitive creation and manipulation of elastoplastic objects in Virtual Reality (VR). By customizing the Material Point Method (MPM) for real-time simulation of hand-induced large deformations and enhancing 3D Gaussian Splatting for seamless rendering, VR-Doh provides an interactive and immersive 3D modeling experience. Users can naturally sculpt, deform, and edit objects through both contact- and gesture-based hand-object interactions. To achieve real-time performance, our system incorporates localized simulation techniques, particle-level collision handling, and the decoupling of physical and appearance representations, ensuring smooth and responsive interactions. VR-Doh supports both object creation and editing, enabling diverse modeling tasks such as designing food items, characters, and interlocking structures, all resulting in simulation-ready assets. User studies with both novice and experienced participants highlight the system's intuitive design, immersive feedback, and creative potential. Compared to existing geometric modeling tools, VR-Doh offers enhanced accessibility and natural interaction, making it a powerful tool for creative exploration in VR.