Adapting Deep Visuomotor Representations with Weak Pairwise Constraints

TL;DR

This paper introduces a novel domain adaptation method for robotic perception that combines distribution and pairwise alignment using weakly paired images, enabling effective transfer from synthetic to real-world data without manual annotation.

Contribution

It proposes a new domain adaptation approach that leverages weakly aligned image pairs to improve visual representation transfer from simulation to real-world robotics tasks.

Findings

Outperforms previous methods in real-world robot manipulation tasks

Effectively compensates for domain shift using weakly paired images

Enables better robot performance without manual data annotation

Abstract

Real-world robotics problems often occur in domains that differ significantly from the robot's prior training environment. For many robotic control tasks, real world experience is expensive to obtain, but data is easy to collect in either an instrumented environment or in simulation. We propose a novel domain adaptation approach for robot perception that adapts visual representations learned on a large easy-to-obtain source dataset (e.g. synthetic images) to a target real-world domain, without requiring expensive manual data annotation of real world data before policy search. Supervised domain adaptation methods minimize cross-domain differences using pairs of aligned images that contain the same object or scene in both the source and target domains, thus learning a domain-invariant representation. However, they require manual alignment of such image pairs. Fully unsupervised adaptation methods rely on minimizing the discrepancy between the feature distributions across domains. We propose a novel, more powerful combination of both distribution and pairwise image alignment, and remove the requirement for expensive annotation by using weakly aligned pairs of images in the source and target domains. Focusing on adapting from simulation to real world data using a PR2 robot, we evaluate our approach on a manipulation task and show that by using weakly paired images, our method compensates for domain shift more effectively than previous techniques, enabling better robot performance in the real world.