FlowBot++: Learning Generalized Articulated Objects Manipulation via Articulation Projection

TL;DR

FlowBot++ is a deep vision-based system that learns to manipulate novel articulated objects by predicting dense motion and articulation parameters, demonstrating high accuracy and generalizability in simulation and real-world tests.

Contribution

It introduces a novel per-point representation for articulated motion and parameters, enhancing the accuracy and generalizability of robotic manipulation of articulated objects.

Findings

Outperforms existing methods in simulated experiments.

Successfully manipulates real-world objects with a Sawyer robot.

Demonstrates generalization to unseen objects and scenarios.

Abstract

Understanding and manipulating articulated objects, such as doors and drawers, is crucial for robots operating in human environments. We wish to develop a system that can learn to articulate novel objects with no prior interaction, after training on other articulated objects. Previous approaches for articulated object manipulation rely on either modular methods which are brittle or end-to-end methods, which lack generalizability. This paper presents FlowBot++, a deep 3D vision-based robotic system that predicts dense per-point motion and dense articulation parameters of articulated objects to assist in downstream manipulation tasks. FlowBot++ introduces a novel per-point representation of the articulated motion and articulation parameters that are combined to produce a more accurate estimate than either method on their own. Simulated experiments on the PartNet-Mobility dataset validate the performance of our system in articulating a wide range of objects, while real-world experiments on real objects' point clouds and a Sawyer robot demonstrate the generalizability and feasibility of our system in real-world scenarios.