Continuously Improving Mobile Manipulation with Autonomous Real-World RL

TL;DR

This paper introduces an autonomous reinforcement learning framework enabling mobile robots to learn and improve manipulation skills in real-world settings without human supervision, using task-relevant exploration and semantic rewards.

Contribution

The authors propose a novel autonomous RL approach that combines task-relevant exploration, behavior priors, and semantic rewards for mobile manipulation in real-world environments.

Findings

Achieved an average success rate of 80% across four manipulation tasks.

Demonstrated continuous improvement in robot performance over time.

Outperformed existing methods by 3-4 times in success rate.

Abstract

We present a fully autonomous real-world RL framework for mobile manipulation that can learn policies without extensive instrumentation or human supervision. This is enabled by 1) task-relevant autonomy, which guides exploration towards object interactions and prevents stagnation near goal states, 2) efficient policy learning by leveraging basic task knowledge in behavior priors, and 3) formulating generic rewards that combine human-interpretable semantic information with low-level, fine-grained observations. We demonstrate that our approach allows Spot robots to continually improve their performance on a set of four challenging mobile manipulation tasks, obtaining an average success rate of 80% across tasks, a 3-4 improvement over existing approaches. Videos can be found at https://continual-mobile-manip.github.io/