Opening Articulated Structures in the Real World

TL;DR

This paper investigates the challenges of enabling mobile robots to open unseen articulated objects in real-world settings, revealing that modular systems outperform end-to-end learning and perception is the main bottleneck.

Contribution

It presents a comprehensive system for opening articulated structures in real environments and provides large-scale empirical insights into system design and perception challenges.

Findings

Modular systems outperform end-to-end learned systems.

Perception is the primary bottleneck, not end-effector control.

Articulation parameter estimation models struggle with robot-centric viewpoints.

Abstract

What does it take to build mobile manipulation systems that can competently operate on previously unseen objects in previously unseen environments? This work answers this question using opening of articulated structures as a mobile manipulation testbed. Specifically, our focus is on the end-to-end performance on this task without any privileged information, i.e. the robot starts at a location with the novel target articulated object in view, and has to approach the object and successfully open it. We first develop a system for this task, and then conduct 100+ end-to-end system tests across 13 real world test sites. Our large-scale study reveals a number of surprising findings: a) modular systems outperform end-to-end learned systems for this task, even when the end-to-end learned systems are trained on 1000+ demonstrations, b) perception, and not precise end-effector control, is the primary bottleneck to task success, and c) state-of-the-art articulation parameter estimation models developed in isolation struggle when faced with robot-centric viewpoints. Overall, our findings highlight the limitations of developing components of the pipeline in isolation and underscore the need for system-level research, providing a pragmatic roadmap for building generalizable mobile manipulation systems. Videos, code, and models are available on the project website: https://arjung128.github.io/opening-articulated-structures/