Accurate Vision-based Manipulation through Contact Reasoning

TL;DR

This paper introduces a novel vision-based manipulation method that improves contact planning efficiency and accuracy by disentangling contact from motion optimization and combining neural networks with physical state representations.

Contribution

It presents a new approach that separates contact planning from motion optimization and integrates neural networks with physical models for better perception and state estimation.

Findings

More efficient contact planning in simulation and real-world

Higher manipulation accuracy compared to previous methods

Effective in partially observable environments

Abstract

Planning contact interactions is one of the core challenges of many robotic tasks. Optimizing contact locations while taking dynamics into account is computationally costly and, in environments that are only partially observable, executing contact-based tasks often suffers from low accuracy. We present an approach that addresses these two challenges for the problem of vision-based manipulation. First, we propose to disentangle contact from motion optimization. Thereby, we improve planning efficiency by focusing computation on promising contact locations. Second, we use a hybrid approach for perception and state estimation that combines neural networks with a physically meaningful state representation. In simulation and real-world experiments on the task of planar pushing, we show that our method is more efficient and achieves a higher manipulation accuracy than previous vision-based approaches.