Learning-Based Strategy for Composite Robot Assembly Skill Adaptation

TL;DR

This paper introduces a modular, reinforcement learning-based strategy for improving contact-rich robot assembly tasks, emphasizing safety, reusability, and robustness in simulation for industrial applications.

Contribution

It presents a residual reinforcement learning approach with composite skills for peg-in-hole assembly, enhancing adaptability and safety while maintaining a structured execution flow.

Findings

Enables robust execution of assembly skills in simulation.

Promotes safety and sample efficiency through residual RL.

Demonstrates effectiveness on a UR5e robot in MuJoCo.

Abstract

Contact-rich robotic skills remain challenging for industrial robots due to tight geometric tolerances, frictional variability, and uncertain contact dynamics, particularly when using position-controlled manipulators. This paper presents a reusable and encapsulated skill-based strategy for peg-in-hole assembly, in which adaptation is achieved through Residual Reinforcement Learning (RRL). The assembly process is represented using composite skills with explicit pre-, post-, and invariant conditions, enabling modularity, reusability, and well-defined execution semantics across task variations. Safety and sample efficiency are promoted through RRL by restricting adaptation to residual refinements within each skill during contact-rich interactions, while the overall skill structure and execution flow remain invariant. The proposed approach is evaluated in MuJoCo simulation on a UR5e robot equipped with a Robotiq gripper and trained using SAC and JAX. Results demonstrate that the proposed formulation enables robust execution of assembly skills, highlighting its suitability for industrial automation.