Adapt Your Body: Mitigating Proprioception Shifts in Imitation Learning

TL;DR

This paper introduces a domain adaptation method using Wasserstein distance to mitigate proprioception shift in imitation learning, improving robotic manipulation performance by aligning training and deployment proprioceptive distributions.

Contribution

It proposes a novel Wasserstein distance-based domain adaptation framework that enhances robustness to proprioception shifts in imitation learning for robotics.

Findings

Outperforms naive proprioception discarding methods

Reduces distributional discrepancy between training and deployment

Improves manipulation task success rates

Abstract

Imitation learning models for robotic tasks typically rely on multi-modal inputs, such as RGB images, language, and proprioceptive states. While proprioception is intuitively important for decision-making and obstacle avoidance, simply incorporating all proprioceptive states leads to a surprising degradation in imitation learning performance. In this work, we identify the underlying issue as the proprioception shift problem, where the distributions of proprioceptive states diverge significantly between training and deployment. To address this challenge, we propose a domain adaptation framework that bridges the gap by utilizing rollout data collected during deployment. Using Wasserstein distance, we quantify the discrepancy between expert and rollout proprioceptive states and minimize this gap by adding noise to both sets of states, proportional to the Wasserstein distance. This strategy enhances robustness against proprioception shifts by aligning the training and deployment distributions. Experiments on robotic manipulation tasks demonstrate the efficacy of our method, enabling the imitation policy to leverage proprioception while mitigating its adverse effects. Our approach outperforms the naive solution which discards proprioception, and other baselines designed to address distributional shifts.