Invariance Through Latent Alignment

TL;DR

The paper introduces Invariance Through Latent Alignment (ILA), a method for unsupervised, deployment-time adaptation of visuomotor policies to unknown perceptual changes, improving robustness without prior knowledge of environment shifts.

Contribution

ILA is a novel approach that aligns latent feature distributions during deployment, enabling better adaptation to unseen perceptual variations in robotic control tasks.

Findings

ILA improves performance under lighting, scene content, and camera pose changes.

Effective in simulation and real-world robot experiments.

Outperforms baseline adaptation methods.

Abstract

A robot's deployment environment often involves perceptual changes that differ from what it has experienced during training. Standard practices such as data augmentation attempt to bridge this gap by augmenting source images in an effort to extend the support of the training distribution to better cover what the agent might experience at test time. In many cases, however, it is impossible to know test-time distribution-shift a priori, making these schemes infeasible. In this paper, we introduce a general approach, called Invariance Through Latent Alignment (ILA), that improves the test-time performance of a visuomotor control policy in deployment environments with unknown perceptual variations. ILA performs unsupervised adaptation at deployment-time by matching the distribution of latent features on the target domain to the agent's prior experience, without relying on paired data. Although simple, we show that this idea leads to surprising improvements on a variety of challenging adaptation scenarios, including changes in lighting conditions, the content in the scene, and camera poses. We present results on calibrated control benchmarks in simulation -- the distractor control suite -- and a physical robot under a sim-to-real setup.