Learning to Manipulate Anywhere: A Visual Generalizable Framework For Reinforcement Learning

TL;DR

This paper introduces Maniwhere, a visual reinforcement learning framework that enhances robot generalization across diverse visual disturbances using multi-view learning, spatial transformers, and curriculum-based augmentation, demonstrating superior sim2real transfer.

Contribution

The paper presents Maniwhere, a novel framework combining multi-view representation learning with spatial transformers and curriculum augmentation to improve visual generalization in reinforcement learning for robots.

Findings

Maniwhere outperforms existing methods in diverse manipulation tasks.

The framework achieves strong sim2real transfer across multiple hardware platforms.

It effectively generalizes across various visual disturbances and viewpoints.

Abstract

Can we endow visuomotor robots with generalization capabilities to operate in diverse open-world scenarios? In this paper, we propose \textbf{Maniwhere}, a generalizable framework tailored for visual reinforcement learning, enabling the trained robot policies to generalize across a combination of multiple visual disturbance types. Specifically, we introduce a multi-view representation learning approach fused with Spatial Transformer Network (STN) module to capture shared semantic information and correspondences among different viewpoints. In addition, we employ a curriculum-based randomization and augmentation approach to stabilize the RL training process and strengthen the visual generalization ability. To exhibit the effectiveness of Maniwhere, we meticulously design 8 tasks encompassing articulate objects, bi-manual, and dexterous hand manipulation tasks, demonstrating Maniwhere's strong visual generalization and sim2real transfer abilities across 3 hardware platforms. Our experiments show that Maniwhere significantly outperforms existing state-of-the-art methods. Videos are provided at https://gemcollector.github.io/maniwhere/.