An Improved Algorithm of Robot Path Planning in Complex Environment Based on Double DQN

TL;DR

This paper introduces an improved Double DQN algorithm for robot path planning in complex environments, integrating A* and RRT strategies to enhance learning efficiency and obstacle avoidance capabilities.

Contribution

The paper proposes a novel DDQN-based method that incorporates RRT initialization and A*-based reward design to improve path planning in complex environments.

Findings

Enhanced obstacle avoidance demonstrated in simulations

Faster learning convergence compared to standard DQN

Successful path planning in complex scenarios

Abstract

Deep Q Network (DQN) has several limitations when applied in planning a path in environment with a number of dilemmas according to our experiment. The reward function may be hard to model, and successful experience transitions are difficult to find in experience replay. In this context, this paper proposes an improved Double DQN (DDQN) to solve the problem by reference to A* and Rapidly-Exploring Random Tree (RRT). In order to achieve the rich experiments in experience replay, the initialization of robot in each training round is redefined based on RRT strategy. In addition, reward for the free positions is specially designed to accelerate the learning process according to the definition of position cost in A*. The simulation experimental results validate the efficiency of the improved DDQN, and robot could successfully learn the ability of obstacle avoidance and optimal path planning in which DQN or DDQN has no effect.