Hybrid of representation learning and reinforcement learning for dynamic and complex robotic motion planning

TL;DR

This paper presents a hybrid deep reinforcement learning approach combining attention mechanisms, skip connections, and LSTM pooling to enhance robotic motion planning in dynamic environments, achieving better convergence and performance.

Contribution

It introduces LSA-DSAC, a novel hybrid algorithm that integrates attention-based reinforcement learning with techniques to mitigate overfitting and improve convergence speed.

Findings

Outperforms state-of-the-art methods in training and evaluation

Successfully implemented and tested on a physical robot in real-world scenarios

Attention network outperforms graph network in environmental state interpretation

Abstract

Motion planning is the soul of robot decision making. Classical planning algorithms like graph search and reaction-based algorithms face challenges in cases of dense and dynamic obstacles. Deep learning algorithms generate suboptimal one-step predictions that cause many collisions. Reinforcement learning algorithms generate optimal or near-optimal time-sequential predictions. However, they suffer from slow convergence, suboptimal converged results, and overfittings. This paper introduces a hybrid algorithm for robotic motion planning: long short-term memory (LSTM) pooling and skip connection for attention-based discrete soft actor critic (LSA-DSAC). First, graph network (relational graph) and attention network (attention weight) interpret the environmental state for the learning of the discrete soft actor critic algorithm. The expressive power of attention network outperforms that of graph in our task by difference analysis of these two representation methods. However, attention based DSAC faces the overfitting problem in training. Second, the skip connection method is integrated to attention based DSAC to mitigate overfitting and improve convergence speed. Third, LSTM pooling is taken to replace the sum operator of attention weigh and eliminate overfitting by slightly sacrificing convergence speed at early-stage training. Experiments show that LSA-DSAC outperforms the state-of-the-art in training and most evaluations. The physical robot is also implemented and tested in the real world.