Real2Sim or Sim2Real: Robotics Visual Insertion using Deep Reinforcement Learning and Real2Sim Policy Adaptation

TL;DR

This paper introduces a visual reinforcement learning approach for robotic insertion tasks and proposes a novel Real2Sim policy adaptation strategy that simplifies sim2real transfer without heavy rendering or domain randomization.

Contribution

The work presents a pure visual RL solution for insertion tasks and introduces the innovative Real2Sim strategy for easier policy adaptation from real to simulation.

Findings

Effective visual RL for robotic insertion achieved

Real2Sim strategy simplifies sim2real transfer

Compared advantages over traditional Sim2Real methods

Abstract

Reinforcement learning has shown a wide usage in robotics tasks, such as insertion and grasping. However, without a practical sim2real strategy, the policy trained in simulation could fail on the real task. There are also wide researches in the sim2real strategies, but most of those methods rely on heavy image rendering, domain randomization training, or tuning. In this work, we solve the insertion task using a pure visual reinforcement learning solution with minimum infrastructure requirement. We also propose a novel sim2real strategy, Real2Sim, which provides a novel and easier solution in policy adaptation. We discuss the advantage of Real2Sim compared with Sim2Real.