Local Connection Reinforcement Learning Method for Efficient Control of Robotic Peg-in-Hole Assembly

TL;DR

This paper introduces a local connection reinforcement learning method for robotic peg-in-hole assembly, improving training efficiency, stability, and speed by focusing on relevant state-action relationships.

Contribution

It proposes a novel LCRL approach using a connection graph to reduce irrelevant state influence, enhancing data efficiency and training outcomes.

Findings

LCRL improves control stability and speed.

LCRL increases training data efficiency.

LCRL achieves higher final rewards in simulations and experiments.

Abstract

Traditional control methods of robotic peg-in-hole assembly rely on complex contact state analysis. Reinforcement learning (RL) is gradually becoming a preferred method of controlling robotic peg-in-hole assembly tasks. However, the training process of RL is quite time-consuming because RL methods are always globally connected, which means all state components are assumed to be the input of policies for all action components, thus increasing action space and state space to be explored. In this paper, we first define continuous space serialized Shapley value (CS3) and construct a connection graph to clarify the correlativity of action components on state components. Then we propose a local connection reinforcement learning (LCRL) method based on the connection graph, which eliminates the influence of irrelevant state components on the selection of action components. The simulation and experiment results demonstrate that the control strategy obtained through LCRL method improves the stability and rapidity of the control process. LCRL method will enhance the data-efficiency and increase the final reward of the training process.