MapIO: Embodied Interaction for the Accessibility of Tactile Maps Through Augmented Touch Exploration and Conversation
Matteo Manzoni, Sergio Mascetti, Dragan Ahmetovic, Ryan Crabb, James M. Coughlan

TL;DR
This paper presents MapIO, an embodied interaction system that enhances tactile maps for visually impaired users by integrating conversational AI to provide detailed, on-demand spatial information through augmented touch and speech.
Contribution
It introduces a novel interaction paradigm combining tactile exploration with conversational AI, enabling accessible, rich spatial data communication for visually impaired users.
Findings
Users found the system usable and engaging.
The system effectively answered diverse spatial questions.
Participants showed increased confidence in map understanding.
Abstract
For individuals who are blind or have low vision, tactile maps provide essential spatial information but are limited in the amount of data they can convey. Digitally augmented tactile maps enhance these capabilities with audio feedback, thereby combining the tactile feedback provided by the map with an audio description of the touched elements. In this context, we explore an embodied interaction paradigm to augment tactile maps with conversational interaction based on Large Language Models, thus enabling users to obtain answers to arbitrary questions regarding the map. We analyze the type of questions the users are interested in asking, engineer the Large Language Model's prompt to provide reliable answers, and study the resulting system with a set of 10 participants, evaluating how the users interact with the system, its usability, and user experience.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTactile and Sensory Interactions
