Mobile Manipulation Leveraging Multiple Views

TL;DR

This paper presents a mobile manipulation system that combines novel navigation and shape completion techniques, enabling robots to accurately grasp objects without localization by using multiple views and uncertainty estimation.

Contribution

It introduces a system that leverages multiple views and shape prediction to improve mobile manipulation, without relying on localization.

Findings

Effective in simulation with real-world data

Improves shape estimation accuracy over single images

Demonstrates successful manipulation without localization

Abstract

While both navigation and manipulation are challenging topics in isolation, many tasks require the ability to both navigate and manipulate in concert. To this end, we propose a mobile manipulation system that leverages novel navigation and shape completion methods to manipulate an object with a mobile robot. Our system utilizes uncertainty in the initial estimation of a manipulation target to calculate a predicted next-best-view. Without the need of localization, the robot then uses the predicted panoramic view at the next-best-view location to navigate to the desired location, capture a second view of the object, create a new model that predicts the shape of object more accurately than a single image alone, and uses this model for grasp planning. We show that the system is highly effective for mobile manipulation tasks through simulation experiments using real world data, as well as ablations on each component of our system.