Virtual Experience to Real World Application: Sidewalk Obstacle Avoidance Using Reinforcement Learning for Visually Impaired

TL;DR

This paper presents a reinforcement learning-based system for sidewalk obstacle avoidance and conversational assistance, integrating simulated training and real-world testing to enhance navigation safety for visually impaired individuals.

Contribution

It introduces a novel integrated prototype combining obstacle avoidance and conversational agents for improved sidewalk navigation safety.

Findings

Prototype improved obstacle avoidance success rate by 5%

Integrated sensory inputs enhance navigation safety

Reinforcement learning effectively trains navigation agents in simulated environments

Abstract

Finding a path free from obstacles that poses minimal risk is critical for safe navigation. People who are sighted and people who are visually impaired require navigation safety while walking on a sidewalk. In this research we developed an assistive navigation on a sidewalk by integrating sensory inputs using reinforcement learning. We trained a Sidewalk Obstacle Avoidance Agent (SOAA) through reinforcement learning in a simulated robotic environment. A Sidewalk Obstacle Conversational Agent (SOCA) is built by training a natural language conversation agent with real conversation data. The SOAA along with SOCA was integrated in a prototype device called augmented guide (AG). Empirical analysis showed that this prototype improved the obstacle avoidance experience about 5% from a base case of 81.29%