Context-Based Meta Reinforcement Learning for Robust and Adaptable Peg-in-Hole Assembly Tasks

TL;DR

This paper introduces a context-based meta reinforcement learning approach for peg-in-hole assembly tasks, improving adaptability, sample efficiency, and generalization in unknown environments with sensor noise and out-of-distribution parameters.

Contribution

It enhances meta RL for PiH assembly by utilizing robot kinematics, uncalibrated cameras, and force/torque data, along with a new adaptation method for out-of-distribution tasks.

Findings

Improved sample efficiency during meta training.

Enhanced real-world adaptation performance.

Better generalization to new task parameters.

Abstract

Autonomous assembly is an essential capability for industrial and service robots, with Peg-in-Hole (PiH) insertion being one of the core tasks. However, PiH assembly in unknown environments is still challenging due to uncertainty in task parameters, such as the hole position and orientation, resulting from sensor noise. Although context-based meta reinforcement learning (RL) methods have been previously presented to adapt to unknown task parameters in PiH assembly tasks, the performance depends on a sample-inefficient procedure or human demonstrations. Thus, to enhance the applicability of meta RL in real-world PiH assembly tasks, we propose to train the agent to use information from the robot's forward kinematics and an uncalibrated camera. Furthermore, we improve the performance by efficiently adapting the meta-trained agent to use data from force/torque sensor. Finally, we propose an adaptation procedure for out-of-distribution tasks whose parameters are different from the training tasks. Experiments on simulated and real robots prove that our modifications enhance the sample efficiency during meta training, real-world adaptation performance, and generalization of the context-based meta RL agent in PiH assembly tasks compared to previous approaches.