ArtiBench and ArtiBrain: Benchmarking Generalizable Vision-Language Articulated Object Manipulation

TL;DR

This paper introduces ArtiBench, a comprehensive benchmark for evaluating generalizable vision-language manipulation of articulated objects, and proposes ArtiBrain, a modular framework that combines reasoning and adaptive control to improve manipulation robustness and generalization.

Contribution

We present ArtiBench, a new benchmark for structured evaluation of articulated object manipulation, and ArtiBrain, a novel framework integrating reasoning, control, and memory for enhanced generalization.

Findings

ArtiBrain outperforms existing methods in robustness and generalization on ArtiBench.

The benchmark reveals key challenges in cross-part and cross-instance manipulation.

The framework effectively combines high-level reasoning with low-level adaptive control.

Abstract

Interactive articulated manipulation requires long-horizon, multi-step interactions with appliances while maintaining physical consistency. Existing vision-language and diffusion-based policies struggle to generalize across parts, instances, and categories. We first introduce ArtiBench, a five-level benchmark covering kitchen, storage, office, and tool environments. ArtiBench enables structured evaluation from cross-part and cross-instance variation to long-horizon multi-object tasks, revealing the core generalization challenges of articulated object manipulation. Building on this benchmark, we propose ArtiBrain, a modular framework that unifies high-level reasoning with adaptive low-level control. ArtiBrain uses a VLM-based Task Reasoner (GPT-4.1) to decompose and validate subgoals, and employs a Hybrid Controller that combines geometry-aware keyframe execution with affordance-guided diffusion for precise and interpretable manipulation. An Affordance Memory Bank continually accumulates successful execution episodes and propagates part-level actionable affordances to unseen articulated parts and configurations. Extensive experiments on ArtiBench show that our ArtiBrain significantly outperforms state-of-the-art multimodal and diffusion-based methods in robustness and generalization. Code and dataset will be released upon acceptance.