Sight Guide: A Wearable Assistive Perception and Navigation System for the Vision Assistance Race in the Cybathlon 2024

TL;DR

Sight Guide is a wearable system that combines robotics algorithms and learning methods to assist visually impaired users in navigation and scene understanding, demonstrated successfully in the Cybathlon 2024 competition.

Contribution

This work introduces a novel wearable assistive system integrating classical and learning-based algorithms for complex perception and navigation tasks in real-world scenarios.

Findings

Achieved 95.7% task success rate in testing environment

Demonstrated effectiveness during Cybathlon competition

Provides detailed system design and evaluation insights

Abstract

Visually impaired individuals face significant challenges navigating and interacting with unknown situations, particularly in tasks requiring spatial awareness and semantic scene understanding. To accelerate the development and evaluate the state of technologies that enable visually impaired people to solve these tasks, the Vision Assistance Race (VIS) at the Cybathlon 2024 competition was organized. In this work, we present Sight Guide, a wearable assistive system designed for the VIS. The system processes data from multiple RGB and depth cameras on an embedded computer that guides the user through complex, real-world-inspired tasks using vibration signals and audio commands. Our software architecture integrates classical robotics algorithms with learning-based approaches to enable capabilities such as obstacle avoidance, object detection, optical character recognition, and touchscreen interaction. In a testing environment, Sight Guide achieved a 95.7% task success rate, and further demonstrated its effectiveness during the Cybathlon competition. This work provides detailed insights into the system design, evaluation results, and lessons learned, and outlines directions towards a broader real-world applicability.