Deep Reinforcement Learning-based Obstacle Avoidance for Robot Movement in Warehouse Environments

TL;DR

This paper introduces a deep reinforcement learning approach for mobile robot obstacle avoidance in warehouses, enhancing learning through pedestrian interaction modeling and spatial behavior-based reward design, validated via simulations.

Contribution

It proposes a novel deep reinforcement learning algorithm that incorporates pedestrian interaction and spatial behavior analysis for improved obstacle avoidance in warehouse robots.

Findings

Enhanced obstacle avoidance performance in complex warehouse environments.

Effective modeling of pedestrian interactions improves robot decision-making.

Simulation results confirm the algorithm's feasibility and robustness.

Abstract

At present, in most warehouse environments, the accumulation of goods is complex, and the management personnel in the control of goods at the same time with the warehouse mobile robot trajectory interaction, the traditional mobile robot can not be very good on the goods and pedestrians to feed back the correct obstacle avoidance strategy, in order to control the mobile robot in the warehouse environment efficiently and friendly to complete the obstacle avoidance task, this paper proposes a deep reinforcement learning based on the warehouse environment, the mobile robot obstacle avoidance Algorithm. Firstly, for the insufficient learning ability of the value function network in the deep reinforcement learning algorithm, the value function network is improved based on the pedestrian interaction, the interaction information between pedestrians is extracted through the pedestrian angle grid, and the temporal features of individual pedestrians are extracted through the attention mechanism, so that we can learn to obtain the relative importance of the current state and the historical trajectory state as well as the joint impact on the robot's obstacle avoidance strategy, which provides an opportunity for the learning of multi-layer perceptual machines afterwards. Secondly, the reward function of reinforcement learning is designed based on the spatial behaviour of pedestrians, and the robot is punished for the state where the angle changes too much, so as to achieve the requirement of comfortable obstacle avoidance; Finally, the feasibility and effectiveness of the deep reinforcement learning-based mobile robot obstacle avoidance algorithm in the warehouse environment in the complex environment of the warehouse are verified through simulation experiments.