Multimodal Feedback for Handheld Tool Guidance: Combining Wrist-Based Haptics with Augmented Reality
Yue Yang, Christoph Leuze, Brian Hargreaves, Bruce Daniel, Fred M Baik

TL;DR
This study demonstrates that combining wrist-based haptic feedback with augmented reality significantly improves spatial precision and user confidence in handheld tool guidance, especially in complex visual environments.
Contribution
The paper introduces a multimodal system integrating wrist haptics with AR, validated through experiments showing enhanced precision and usability in surgical guidance tasks.
Findings
Haptic cues improved spatial accuracy to 5.8 mm.
Participants reported higher usability scores (SUS=88.1).
Haptics reduced cognitive effort during tool alignment.
Abstract
We investigate how vibrotactile wrist feedback can enhance spatial guidance for handheld tool movement in optical see-through augmented reality (AR). While AR overlays are widely used to support surgical tasks, visual occlusion, lighting conditions, and interface ambiguity can compromise precision and confidence. To address these challenges, we designed a multimodal system combining AR visuals with a custom wrist-worn haptic device delivering directional and state-based cues. A formative study with experienced surgeons and residents identified key tool maneuvers and preferences for reference mappings, guiding our cue design. In a cue identification experiment (N=21), participants accurately recognized five vibration patterns under visual load, with higher recognition for full-actuator states than spatial direction cues. In a guidance task (N=27), participants using both AR and haptics…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTactile and Sensory Interactions · Augmented Reality Applications · Teleoperation and Haptic Systems
