A Study on Enhancing the Generalization Ability of Visuomotor Policies via Data Augmentation

TL;DR

This paper investigates how extensive scene randomization during data generation improves the generalization of visuomotor policies, especially for zero-shot sim-to-real transfer, by creating a diverse dataset with minimal human effort.

Contribution

It introduces a highly randomized dataset covering multiple manipulators and factors, demonstrating that diverse data enhances policy generalization and transferability.

Findings

Randomization factors significantly impact policy generalization.

Diverse trajectories effectively bridge visual gaps.

Scene randomization improves zero-shot sim-to-real transfer.

Abstract

The generalization ability of visuomotor policy is crucial, as a good policy should be deployable across diverse scenarios. Some methods can collect large amounts of trajectory augmentation data to train more generalizable imitation learning policies, aimed at handling the random placement of objects on the scene's horizontal plane. However, the data generated by these methods still lack diversity, which limits the generalization ability of the trained policy. To address this, we investigate the performance of policies trained by existing methods across different scene layout factors via automate the data generation for those factors that significantly impact generalization. We have created a more extensively randomized dataset that can be efficiently and automatically generated with only a small amount of human demonstration. The dataset covers five types of manipulators and two types of grippers, incorporating extensive randomization factors such as camera pose, lighting conditions, tabletop texture, and table height across six manipulation tasks. We found that all of these factors influence the generalization ability of the policy. Applying any form of randomization enhances policy generalization, with diverse trajectories particularly effective in bridging visual gap. Notably, we investigated on low-cost manipulator the effect of the scene randomization proposed in this work on enhancing the generalization capability of visuomotor policies for zero-shot sim-to-real transfer.