Adaptive Environment Modeling Based Reinforcement Learning for Collision Avoidance in Complex Scenes

TL;DR

This paper introduces AEMCARL, an adaptive reinforcement learning framework that combines hierarchical environment modeling, attention-based perception, and adaptive rewards to enable robots to navigate safely in crowded scenes.

Contribution

It proposes a novel hierarchical GRU-based environment model, an attention-driven perception mechanism, and an adaptive reward function for improved collision avoidance.

Findings

Outperforms baseline methods in simulation and real-world tests.

Enables robots to navigate safely in complex crowded environments.

Demonstrates robustness across different robot platforms and scenarios.

Abstract

The major challenges of collision avoidance for robot navigation in crowded scenes lie in accurate environment modeling, fast perceptions, and trustworthy motion planning policies. This paper presents a novel adaptive environment model based collision avoidance reinforcement learning (i.e., AEMCARL) framework for an unmanned robot to achieve collision-free motions in challenging navigation scenarios. The novelty of this work is threefold: (1) developing a hierarchical network of gated-recurrent-unit (GRU) for environment modeling; (2) developing an adaptive perception mechanism with an attention module; (3) developing an adaptive reward function for the reinforcement learning (RL) framework to jointly train the environment model, perception function and motion planning policy. The proposed method is tested with the Gym-Gazebo simulator and a group of robots (Husky and Turtlebot) under various crowded scenes. Both simulation and experimental results have demonstrated the superior performance of the proposed method over baseline methods.