RMMI: Reactive Mobile Manipulation using an Implicit Neural Map

TL;DR

RMMI introduces a neural SDF-based reactive control framework for mobile manipulators, enabling better obstacle avoidance and motion optimization in cluttered environments, demonstrated through simulation and real-world tests.

Contribution

The paper presents RMMI, a novel reactive control method that uses neural Signed Distance Fields for improved environment representation and collision avoidance in mobile manipulation.

Findings

25% increase in success rate in cluttered environments

Effective joint optimization of base and arm motions

Validated in both simulation and real-world experiments

Abstract

Mobile manipulator robots operating in complex domestic and industrial environments must effectively coordinate their base and arm motions while avoiding obstacles. While current reactive control methods gracefully achieve this coordination, they rely on simplified and idealised geometric representations of the environment to avoid collisions. This limits their performance in cluttered environments. To address this problem, we introduce RMMI, a reactive control framework that leverages the ability of neural Signed Distance Fields (SDFs) to provide a continuous and differentiable representation of the environment's geometry. RMMI formulates a quadratic program that optimises jointly for robot base and arm motion, maximises the manipulability, and avoids collisions through a set of inequality constraints. These constraints are constructed by querying the SDF for the distance and direction to the closest obstacle for a large number of sampling points on the robot. We evaluate RMMI both in simulation and in a set of real-world experiments. For reaching in cluttered environments, we observe a 25% increase in success rate. For additional details, code, and experiment videos, please visit https://rmmi.github.io/.