Deep Reinforcement Learning Based Robot Arm Manipulation with Efficient Training Data through Simulation

TL;DR

This paper introduces an adaptive replay buffer update method for deep reinforcement learning, improving robot arm manipulation efficiency in simulation and real-world experiments by selectively storing relevant experiences.

Contribution

It presents a novel adaptive replay buffer update technique that enhances training efficiency and performance in robot manipulation tasks compared to standard methods.

Findings

Improved policy performance over traditional replay buffer methods.

Effective transfer from simulation to real robot experiments.

Enhanced training efficiency with selective experience storage.

Abstract

Deep reinforcement learning trains neural networks using experiences sampled from the replay buffer, which is commonly updated at each time step. In this paper, we propose a method to update the replay buffer adaptively and selectively to train a robot arm to accomplish a suction task in simulation. The response time of the agent is thoroughly taken into account. The state transitions that remain stuck at the boundary of constraint are not stored. The policy trained with our method works better than the one with the common replay buffer update method. The result is demonstrated both by simulation and by experiment with a real robot arm.