A Reconfigurable Data Glove for Reconstructing Physical and Virtual Grasps
Hangxin Liu, Zeyu Zhang, Ziyuan Jiao, Zhenliang Zhang, Minchen Li,, Chenfanfu Jiang, Yixin Zhu, Song-Chun Zhu

TL;DR
This paper introduces a reconfigurable data glove capable of capturing human hand-object interactions across physical, virtual, and simulated environments, facilitating advanced manipulation learning for embodied AI.
Contribution
It presents a novel multi-mode glove design integrating tactile sensing, VR interaction, and high-fidelity simulation, enabling comprehensive data collection for manipulation tasks.
Findings
Effective real-time gesture and force recording
Enhanced manipulation fluency in VR environments
Realistic simulation of tool use and physical interactions
Abstract
In this work, we present a reconfigurable data glove design to capture different modes of human hand-object interactions, which are critical in training embodied artificial intelligence (AI) agents for fine manipulation tasks. To achieve various downstream tasks with distinct features, our reconfigurable data glove operates in three modes sharing a unified backbone design that reconstructs hand gestures in real time. In the tactile-sensing mode, the glove system aggregates manipulation force via customized force sensors made from a soft and thin piezoresistive material; this design minimizes interference during complex hand movements. The virtual reality (VR) mode enables real-time interaction in a physically plausible fashion: A caging-based approach is devised to determine stable grasps by detecting collision events. Leveraging a state-of-the-art finite element method (FEM), the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Sensor and Energy Harvesting Materials · Robot Manipulation and Learning · Muscle activation and electromyography studies
MethodsGloVe Embeddings
