DemoStart: Demonstration-led auto-curriculum applied to sim-to-real with multi-fingered robots

TL;DR

DemoStart is an auto-curriculum reinforcement learning method that enables complex multi-fingered robot manipulation from sparse rewards and minimal demonstrations, achieving effective sim-to-real transfer directly from raw pixel inputs.

Contribution

It introduces a demonstration-led auto-curriculum approach that significantly reduces demonstration requirements and enhances zero-shot sim-to-real transfer for complex robotic manipulation.

Findings

Outperforms demonstration-only policies on real robots.

Requires 100 times fewer demonstrations than traditional methods.

Successfully transfers policies directly from simulation to real-world using raw pixels.

Abstract

We present DemoStart, a novel auto-curriculum reinforcement learning method capable of learning complex manipulation behaviors on an arm equipped with a three-fingered robotic hand, from only a sparse reward and a handful of demonstrations in simulation. Learning from simulation drastically reduces the development cycle of behavior generation, and domain randomization techniques are leveraged to achieve successful zero-shot sim-to-real transfer. Transferred policies are learned directly from raw pixels from multiple cameras and robot proprioception. Our approach outperforms policies learned from demonstrations on the real robot and requires 100 times fewer demonstrations, collected in simulation. More details and videos in https://sites.google.com/view/demostart.