Cross-Domain Policy Transfer by Representation Alignment via Multi-Domain Behavioral Cloning

TL;DR

This paper introduces a simple yet effective method for cross-domain policy transfer that learns a shared latent space and an abstract policy using multi-domain behavioral cloning and MMD regularization, outperforming prior domain translation approaches.

Contribution

It proposes a novel approach combining multi-domain behavioral cloning with MMD regularization to improve cross-domain policy transfer, especially under significant domain gaps.

Findings

Higher transfer performance with MMD regularization.

Effective in cross-morphology and cross-viewpoint scenarios.

Single multi-domain policy simplifies extension.

Abstract

Transferring learned skills across diverse situations remains a fundamental challenge for autonomous agents, particularly when agents are not allowed to interact with an exact target setup. While prior approaches have predominantly focused on learning domain translation, they often struggle with handling significant domain gaps or out-of-distribution tasks. In this paper, we present a simple approach for cross-domain policy transfer that learns a shared latent representation across domains and a common abstract policy on top of it. Our approach leverages multi-domain behavioral cloning on unaligned trajectories of proxy tasks and employs maximum mean discrepancy (MMD) as a regularization term to encourage cross-domain alignment. The MMD regularization better preserves structures of latent state distributions than commonly used domain-discriminative distribution matching, leading to higher transfer performance. Moreover, our approach involves training only one multi-domain policy, which makes extension easier than existing methods. Empirical evaluations demonstrate the efficacy of our method across various domain shifts, especially in scenarios where exact domain translation is challenging, such as cross-morphology or cross-viewpoint settings. Our ablation studies further reveal that multi-domain behavioral cloning implicitly contributes to representation alignment alongside domain-adversarial regularization.