ScrewSplat: An End-to-End Method for Articulated Object Recognition

TL;DR

ScrewSplat is an end-to-end RGB-based method that accurately recognizes and models articulated objects' geometry and kinematics, enabling improved robotic interaction without relying on additional inputs or assumptions.

Contribution

It introduces a novel RGB-only approach that jointly reconstructs geometry and kinematic structure, surpassing prior methods in accuracy and practicality.

Findings

Achieves state-of-the-art recognition accuracy.

Enables zero-shot, text-guided manipulation.

Operates solely on RGB observations.

Abstract

Articulated object recognition -- the task of identifying both the geometry and kinematic joints of objects with movable parts -- is essential for enabling robots to interact with everyday objects such as doors and laptops. However, existing approaches often rely on strong assumptions, such as a known number of articulated parts; require additional inputs, such as depth images; or involve complex intermediate steps that can introduce potential errors -- limiting their practicality in real-world settings. In this paper, we introduce ScrewSplat, a simple end-to-end method that operates solely on RGB observations. Our approach begins by randomly initializing screw axes, which are then iteratively optimized to recover the object's underlying kinematic structure. By integrating with Gaussian Splatting, we simultaneously reconstruct the 3D geometry and segment the object into rigid, movable parts. We demonstrate that our method achieves state-of-the-art recognition accuracy across a diverse set of articulated objects, and further enables zero-shot, text-guided manipulation using the recovered kinematic model. See the project website at: https://screwsplat.github.io.