When Absolute State Fails: Evaluating Proprioceptive Encodings for Robust Manipulation

TL;DR

This paper evaluates different proprioceptive encoding strategies for robotic manipulation, finding that episode-wise relative frames enhance robustness and performance in diverse, unseen conditions.

Contribution

The study systematically compares proprioceptive encoding methods, identifying episode-wise relative frames as optimal for robustness and generalization in robotic tasks.

Findings

Episode-wise relative frames outperform fixed frames in real-robot tests.

Robustness improves with data collected from varying frames of reference.

Proposed encoding strategy enables better generalization to unseen configurations.

Abstract

As end-to-end robotic policies are progressively deployed in the real world to solve real tasks, they face a gap between the training and inference conditions. Scaling the amount and diversity of the training data has shown some success in improving zero-shot generalization, yet robots still fail when faced with new, unseen test conditions. For instance, while robots with fixed frames of reference are common, those with moving frames pose a greater challenge for deployment. To address this specific instance of the issue, we present a study of strategies for encoding the robot's proprioceptive state to improve both in- and out-of-distribution performance at test time. Through a systematic study of joint representations, we find that a simple episode-wise relative frame provides the best trade-off between task performance and robustness, outperforming the baselines in extensive real-robot experiments conducted in a realistic test environment. The results suggest a practical path to leveraging data collected by robots with varying frames of reference and deployment to unseen test configurations.