NaviNote: Enabling In-situ Spatial Annotation Authoring to Support Exploration and Navigation for Blind and Low Vision People

TL;DR

NaviNote is a high-precision, vision-based annotation system that enhances exploration and navigation for blind and low vision users by enabling accurate environmental annotations and last-few-meters navigation.

Contribution

The paper introduces NaviNote, a novel high-accuracy visual positioning system with voice-based annotation and navigation tailored for BLV users, addressing limitations of existing GPS-based solutions.

Findings

NaviNote significantly improved navigation accuracy for BLV users.

Participants found NaviNote helpful for understanding and annotating surroundings.

High-precision visual localization was effective in real-world exploration.

Abstract

GPS and smartphones enable users to place location-based annotations, capturing rich environmental context. Previous research demonstrates that blind and low vision (BLV) people can use annotations to explore unfamiliar areas. However, current commercial systems allowing BLV users to create annotations have never been evaluated, and current GPS-based systems can deviate several meters. Motivated by high-accuracy visual positioning technology, we first conducted a formative study with 24 BLV participants to envision a more accurate and inclusive annotation system. Surprisingly, many participants viewed the high-accuracy technology not just as an annotation system but also as a tool for precise last-few-meters navigation. Guided by participant feedback, we developed NaviNote, which combines vision-based high-precision localization with an agentic architecture to enable voice-based annotation authoring and navigation. Evaluating NaviNote with 18 BLV participants showed that it significantly improved navigation performance and supported users in understanding and annotating their surroundings. Based on these findings, we discuss design considerations for future accessible annotation authoring systems.