Meta Reinforcement Learning for Sim-to-real Domain Adaptation

TL;DR

This paper introduces a meta reinforcement learning approach combined with trajectory generation to enable efficient sim-to-real transfer for robotic control, improving adaptation stability and performance on real hardware.

Contribution

It proposes a novel meta learning framework with task-specific trajectory generation for rapid sim-to-real domain adaptation in robotics.

Findings

Enhanced domain adaptation stability and consistency

Improved real-world task performance on a robotic hockey task

Effective latent space structure during adaptation

Abstract

Modern reinforcement learning methods suffer from low sample efficiency and unsafe exploration, making it infeasible to train robotic policies entirely on real hardware. In this work, we propose to address the problem of sim-to-real domain transfer by using meta learning to train a policy that can adapt to a variety of dynamic conditions, and using a task-specific trajectory generation model to provide an action space that facilitates quick exploration. We evaluate the method by performing domain adaptation in simulation and analyzing the structure of the latent space during adaptation. We then deploy this policy on a KUKA LBR 4+ robot and evaluate its performance on a task of hitting a hockey puck to a target. Our method shows more consistent and stable domain adaptation than the baseline, resulting in better overall performance.