Online Estimation and Manipulation of Articulated Objects

TL;DR

This paper presents a novel online method that combines visual priors and proprioceptive sensing to estimate and manipulate articulated objects, enabling robots to open unseen drawers with high success.

Contribution

It introduces a factor graph approach that fuses vision and sensing data for real-time articulation estimation during manipulation tasks.

Findings

Achieved 75% success rate in real-world experiments

Enabled robots to open previously unseen drawers

Validated effectiveness in simulation and real-world tests

Abstract

From refrigerators to kitchen drawers, humans interact with articulated objects effortlessly every day while completing household chores. For automating these tasks, service robots must be capable of manipulating arbitrary articulated objects. Recent deep learning methods have been shown to predict valuable priors on the affordance of articulated objects from vision. In contrast, many other works estimate object articulations by observing the articulation motion, but this requires the robot to already be capable of manipulating the object. In this article, we propose a novel approach combining these methods by using a factor graph for online estimation of articulation which fuses learned visual priors and proprioceptive sensing during interaction into an analytical model of articulation based on Screw Theory. With our method, a robotic system makes an initial prediction of articulation from vision before touching the object, and then quickly updates the estimate from kinematic and force sensing during manipulation. We evaluate our method extensively in both simulations and real-world robotic manipulation experiments. We demonstrate several closed-loop estimation and manipulation experiments in which the robot was capable of opening previously unseen drawers. In real hardware experiments, the robot achieved a 75% success rate for autonomous opening of unknown articulated objects.