N$^2$M$^2$: Learning Navigation for Arbitrary Mobile Manipulation Motions in Unseen and Dynamic Environments

TL;DR

N$^2$M$^2$ introduces a novel approach for mobile manipulation that combines task space motion generation with reinforcement learning-based navigation, enabling robots to perform complex, long-horizon tasks in dynamic, unseen environments.

Contribution

It extends previous decomposition methods to handle complex obstacle environments and real-world scenarios, allowing for versatile and reactive mobile manipulation.

Findings

Successfully performed long-horizon tasks in unseen environments

Reacted instantly to dynamic obstacles and environmental changes

Validated on multiple real-world mobile manipulators

Abstract

Despite its importance in both industrial and service robotics, mobile manipulation remains a significant challenge as it requires a seamless integration of end-effector trajectory generation with navigation skills as well as reasoning over long-horizons. Existing methods struggle to control the large configuration space, and to navigate dynamic and unknown environments. In previous work, we proposed to decompose mobile manipulation tasks into a simplified motion generator for the end-effector in task space and a trained reinforcement learning agent for the mobile base to account for kinematic feasibility of the motion. In this work, we introduce Neural Navigation for Mobile Manipulation (N$^2$M$^2$) which extends this decomposition to complex obstacle environments and enables it to tackle a broad range of tasks in real world settings. The resulting approach can perform unseen, long-horizon tasks in unexplored environments while instantly reacting to dynamic obstacles and environmental changes. At the same time, it provides a simple way to define new mobile manipulation tasks. We demonstrate the capabilities of our proposed approach in extensive simulation and real-world experiments on multiple kinematically diverse mobile manipulators. Code and videos are publicly available at http://mobile-rl.cs.uni-freiburg.de.