A Mechanistic Analysis of Sim-and-Real Co-Training in Generative Robot Policies

TL;DR

This paper provides a theoretical and empirical analysis of sim-and-real co-training for generative robot policies, identifying key effects that influence its success and proposing a method to improve it.

Contribution

It uncovers two intrinsic effects—structured representation alignment and importance reweighting—that govern co-training performance and offers a simple method to enhance existing approaches.

Findings

Structured representation alignment is crucial for downstream performance.

Importance reweighting modulates action weighting based on domain.

The proposed method outperforms prior approaches in experiments.

Abstract

Co-training, which combines limited in-domain real-world data with abundant surrogate data such as simulation or cross-embodiment robot data, is widely used for training generative robot policies. Despite its empirical success, the mechanisms that determine when and why co-training is effective remain poorly understood. We investigate the mechanism of sim-and-real co-training through theoretical analysis and empirical study, and identify two intrinsic effects governing performance. The first, \textbf{``structured representation alignment"}, reflects a balance between cross-domain representation alignment and domain discernibility, and plays a primary role in downstream performance. The second, the \textbf{``importance reweighting effect"}, arises from domain-dependent modulation of action weighting and operates at a secondary level. We validate these effects with controlled experiments on a toy model and extensive sim-and-sim and sim-and-real robot manipulation experiments. Our analysis offers a unified interpretation of recent co-training techniques and motivates a simple method that consistently improves upon prior approaches. More broadly, our aim is to examine the inner workings of co-training and to facilitate research in this direction.