Combining Gesture and Voice Control for Mid-Air Manipulation of CAD Models in VR Environments

TL;DR

This paper presents a VR system combining gesture and voice controls to simplify 3D CAD modeling, enhancing user interaction and understanding through an integrated approach.

Contribution

It introduces a hybrid gesture-voice interface for VR CAD modeling, utilizing CSG trees for intuitive object manipulation and addressing gesture recognition limitations.

Findings

Improved user experience in VR CAD modeling

Effective integration of voice commands with gesture controls

Positive user study results demonstrating system usability

Abstract

Modeling 3D objects in domains like Computer Aided Design (CAD) is time-consuming and comes with a steep learning curve needed to master the design process as well as tool complexities. In order to simplify the modeling process, we designed and implemented a prototypical system that leverages the strengths of Virtual Reality (VR) hand gesture recognition in combination with the expressiveness of a voice-based interface for the task of 3D modeling. Furthermore, we use the Constructive Solid Geometry (CSG) tree representation for 3D models within the VR environment to let the user manipulate objects from the ground up, giving an intuitive understanding of how the underlying basic shapes connect. The system uses standard mid-air 3D object manipulation techniques and adds a set of voice commands to help mitigate the deficiencies of current hand gesture recognition techniques. A user study was conducted to evaluate the proposed prototype. The combination of our hybrid input paradigm shows to be a promising step towards easier to use CAD modeling.