Multimodal Fusion for Sim2real Transfer in Visual Reinforcement Learning

TL;DR

This paper introduces a multimodal fusion approach combining RGB and depth data with a vision transformer and contrastive learning to improve sim2real transfer in visual reinforcement learning, demonstrating superior performance and real-world applicability.

Contribution

The paper presents a novel fusion scheme using CNNs and vision transformers with contrastive learning and curriculum-based domain randomization for enhanced generalization in sim2real transfer.

Findings

Outperforms baseline methods in simulation

Achieves successful zero-shot transfer to real-world tasks

Enhances sample efficiency and robustness through contrastive learning

Abstract

Depth information is robust to scene appearance variations and inherently carries 3D spatial details. Thus, a visual backbone based on the vision transformer is proposed to fuse RGB and depth modalities for enhancing generalization in this paper. Different modalities are first processed by separate CNN stems, and the combined convolutional features are delivered to the scalable vision transformer to obtain visual representations. Moreover, a contrastive learning scheme is designed with masked and unmasked tokens to enhance the sample efficiency and generalization performance. A curriculum-based domain randomization scheme is used to flexibly stabilize the training process. Finally, simulation results demonstrate that our fusion scheme outperforms the other baselines. The feasibility of our model is validated to perform real-world manipulation tasks via zero-shot transfer.