Vision-based navigation and obstacle avoidance via deep reinforcement learning

TL;DR

This paper presents a deep reinforcement learning approach using deep Dyna-Q for vision-based robot navigation and obstacle avoidance in complex, dynamic environments with limited prior information, demonstrating effective generalization and collision-free evacuation.

Contribution

The study introduces a novel deep Dyna-Q based method for vision-based navigation that handles dynamic obstacles and complex configurations without detailed maps.

Findings

Successfully navigates to goal avoiding static and dynamic obstacles

Generalizes to complex obstacle configurations

Achieves collision-free evacuation in varied environments

Abstract

Development of navigation algorithms is essential for the successful deployment of robots in rapidly changing hazardous environments for which prior knowledge of configuration is often limited or unavailable. Use of traditional path-planning algorithms, which are based on localization and require detailed obstacle maps with goal locations, is not possible. In this regard, vision-based algorithms hold great promise, as visual information can be readily acquired by a robot's onboard sensors and provides a much richer source of information from which deep neural networks can extract complex patterns. Deep reinforcement learning has been used to achieve vision-based robot navigation. However, the efficacy of these algorithms in environments with dynamic obstacles and high variation in the configuration space has not been thoroughly investigated. In this paper, we employ a deep Dyna-Q learning algorithm for room evacuation and obstacle avoidance in partially observable environments based on low-resolution raw image data from an onboard camera. We explore the performance of a robotic agent in environments containing no obstacles, convex obstacles, and concave obstacles, both static and dynamic. Obstacles and the exit are initialized in random positions at the start of each episode of reinforcement learning. Overall, we show that our algorithm and training approach can generalize learning for collision-free evacuation of environments with complex obstacle configurations. It is evident that the agent can navigate to a goal location while avoiding multiple static and dynamic obstacles, and can escape from a concave obstacle while searching for and navigating to the exit.