Reinforcement Learning from Wild Animal Videos

TL;DR

This paper introduces RLWAV, a novel method that learns quadruped robot locomotion skills by leveraging wild animal videos, bridging the gap between natural animal movements and robotic control through reinforcement learning.

Contribution

The paper presents a new approach to robot learning that uses animal videos as a source of diverse motion data, enabling robots to acquire multiple skills without explicit trajectories or rewards.

Findings

Robots learned diverse skills like walking, jumping, and staying still.

The method successfully transfers policies from simulation to real robots.

Wild animal videos can effectively inform robot locomotion learning.

Abstract

We propose to learn legged robot locomotion skills by watching thousands of wild animal videos from the internet, such as those featured in nature documentaries. Indeed, such videos offer a rich and diverse collection of plausible motion examples, which could inform how robots should move. To achieve this, we introduce Reinforcement Learning from Wild Animal Videos (RLWAV), a method to ground these motions into physical robots. We first train a video classifier on a large-scale animal video dataset to recognize actions from RGB clips of animals in their natural habitats. We then train a multi-skill policy to control a robot in a physics simulator, using the classification score of a third-person camera capturing videos of the robot's movements as a reward for reinforcement learning. Finally, we directly transfer the learned policy to a real quadruped Solo. Remarkably, despite the extreme gap in both domain and embodiment between animals in the wild and robots, our approach enables the policy to learn diverse skills such as walking, jumping, and keeping still, without relying on reference trajectories nor skill-specific rewards.