Policy Transfer across Visual and Dynamics Domain Gaps via Iterative Grounding

TL;DR

This paper introduces IDAPT, an iterative environment grounding method that effectively transfers policies across visual and dynamics domain gaps, enabling robots to adapt from simulated or lab environments to real-world settings with minimal supervision.

Contribution

The paper presents a novel iterative policy transfer approach that simultaneously addresses visual and dynamics domain gaps through environment grounding and policy training.

Findings

Successfully transfers policies across visual and dynamics gaps

Reduces the need for extensive target environment interaction

Demonstrates effectiveness on locomotion and manipulation tasks

Abstract

The ability to transfer a policy from one environment to another is a promising avenue for efficient robot learning in realistic settings where task supervision is not available. This can allow us to take advantage of environments well suited for training, such as simulators or laboratories, to learn a policy for a real robot in a home or office. To succeed, such policy transfer must overcome both the visual domain gap (e.g. different illumination or background) and the dynamics domain gap (e.g. different robot calibration or modelling error) between source and target environments. However, prior policy transfer approaches either cannot handle a large domain gap or can only address one type of domain gap at a time. In this paper, we propose a novel policy transfer method with iterative "environment grounding", IDAPT, that alternates between (1) directly minimizing both visual and dynamics domain gaps by grounding the source environment in the target environment domains, and (2) training a policy on the grounded source environment. This iterative training progressively aligns the domains between the two environments and adapts the policy to the target environment. Once trained, the policy can be directly executed on the target environment. The empirical results on locomotion and robotic manipulation tasks demonstrate that our approach can effectively transfer a policy across visual and dynamics domain gaps with minimal supervision and interaction with the target environment. Videos and code are available at https://clvrai.com/idapt .