Green Screen Augmentation Enables Scene Generalisation in Robotic Manipulation

TL;DR

This paper introduces GreenAug, a green screen-based data augmentation method that significantly improves the generalisation of vision-based robotic manipulation policies across different environments, without novel algorithms.

Contribution

The paper proposes GreenAug, a novel data collection and augmentation approach using green screens to enhance scene generalisation in robotic manipulation.

Findings

GreenAug outperforms standard augmentation methods in real-world tests.

Over 850 demonstrations and 8.2k episodes validate GreenAug's effectiveness.

GreenAug enables policies to generalise to visually distinct environments.

Abstract

Generalising vision-based manipulation policies to novel environments remains a challenging area with limited exploration. Current practices involve collecting data in one location, training imitation learning or reinforcement learning policies with this data, and deploying the policy in the same location. However, this approach lacks scalability as it necessitates data collection in multiple locations for each task. This paper proposes a novel approach where data is collected in a location predominantly featuring green screens. We introduce Green-screen Augmentation (GreenAug), employing a chroma key algorithm to overlay background textures onto a green screen. Through extensive real-world empirical studies with over 850 training demonstrations and 8.2k evaluation episodes, we demonstrate that GreenAug surpasses no augmentation, standard computer vision augmentation, and prior generative augmentation methods in performance. While no algorithmic novelties are claimed, our paper advocates for a fundamental shift in data collection practices. We propose that real-world demonstrations in future research should utilise green screens, followed by the application of GreenAug. We believe GreenAug unlocks policy generalisation to visually distinct novel locations, addressing the current scene generalisation limitations in robot learning.