SketchPlay: Intuitive Creation of Physically Realistic VR Content with Gesture-Driven Sketching
Xiangwen Zhang, Xiaowei Dai, Runnan Chen, Xiaoming Chen, Zeke Zexi Hu

TL;DR
SketchPlay is a VR framework that enables intuitive creation of physically realistic scenes through gesture-driven sketching, making complex content creation accessible for non-experts and enhancing expressiveness and user engagement.
Contribution
It introduces a novel VR interaction method combining sketches and gestures to generate dynamic, physically realistic content, simplifying the creation process for non-expert users.
Findings
Outperforms traditional text-driven methods in expressiveness.
Enhances user experience and engagement.
Enables creation of complex physical phenomena like cloth and elastic dynamics.
Abstract
Creating physically realistic content in VR often requires complex modeling tools or predefined 3D models, textures, and animations, which present significant barriers for non-expert users. In this paper, we propose SketchPlay, a novel VR interaction framework that transforms humans' air-drawn sketches and gestures into dynamic, physically realistic scenes, making content creation intuitive and playful like drawing. Specifically, sketches capture the structure and spatial arrangement of objects and scenes, while gestures convey physical cues such as velocity, direction, and force that define movement and behavior. By combining these complementary forms of input, SketchPlay captures both the structure and dynamics of user-created content, enabling the generation of a wide range of complex physical phenomena, such as rigid body motion, elastic deformation, and cloth dynamics. Experimental…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsInteractive and Immersive Displays · Human Motion and Animation · 3D Shape Modeling and Analysis
