Deep Reinforcement Learning based Robot Navigation in Dynamic Environments using Occupancy Values of Motion Primitives

TL;DR

This paper introduces a deep reinforcement learning approach for robot navigation in dynamic environments, utilizing occupancy-based observations of motion primitives to improve training efficiency and navigation performance, validated in simulation and real-world tests.

Contribution

The paper proposes a novel occupancy observation method for deep reinforcement learning in robot navigation, enhancing training speed and navigation accuracy over traditional data structures.

Findings

Reduces training time significantly.

Improves navigation performance in dynamic environments.

Validated on both simulation and physical robots.

Abstract

This paper presents a Deep Reinforcement Learning based navigation approach in which we define the occupancy observations as heuristic evaluations of motion primitives, rather than using raw sensor data. Our method enables fast mapping of the occupancy data, generated by multi-sensor fusion, into trajectory values in 3D workspace. The computationally efficient trajectory evaluation allows dense sampling of the action space. We utilize our occupancy observations in different data structures to analyze their effects on both training process and navigation performance. We train and test our methodology on two different robots within challenging physics-based simulation environments including static and dynamic obstacles. We benchmark our occupancy representations with other conventional data structures from state-of-the-art methods. The trained navigation policies are also validated successfully with physical robots in dynamic environments. The results show that our method not only decreases the required training time but also improves the navigation performance as compared to other occupancy representations. The open-source implementation of our work and all related info are available at \url{https://github.com/RIVeR-Lab/tentabot}.