MapIO: Embodied Interaction for the Accessibility of Tactile Maps Through Augmented Touch Exploration and Conversation

TL;DR

This paper presents MapIO, an embodied interaction system that enhances tactile maps for visually impaired users by integrating conversational AI to provide detailed, on-demand spatial information through augmented touch and speech.

Contribution

It introduces a novel interaction paradigm combining tactile exploration with conversational AI, enabling accessible, rich spatial data communication for visually impaired users.

Findings

Users found the system usable and engaging.

The system effectively answered diverse spatial questions.

Participants showed increased confidence in map understanding.

Abstract

For individuals who are blind or have low vision, tactile maps provide essential spatial information but are limited in the amount of data they can convey. Digitally augmented tactile maps enhance these capabilities with audio feedback, thereby combining the tactile feedback provided by the map with an audio description of the touched elements. In this context, we explore an embodied interaction paradigm to augment tactile maps with conversational interaction based on Large Language Models, thus enabling users to obtain answers to arbitrary questions regarding the map. We analyze the type of questions the users are interested in asking, engineer the Large Language Model's prompt to provide reliable answers, and study the resulting system with a set of 10 participants, evaluating how the users interact with the system, its usability, and user experience.