BimArt: A Unified Approach for the Synthesis of 3D Bimanual Interaction with Articulated Objects

TL;DR

BimArt introduces a unified generative method for creating realistic 3D bimanual interactions with articulated objects, eliminating the need for reference grasps or separate grasping and articulation modes.

Contribution

It presents a novel contact map-based approach conditioned on object trajectories, improving the realism and diversity of bimanual hand-object interaction synthesis.

Findings

Outperforms state-of-the-art in motion quality and diversity

Generates realistic, diverse bimanual interaction animations

Provides insights into feature representation for articulated objects

Abstract

We present BimArt, a novel generative approach for synthesizing 3D bimanual hand interactions with articulated objects. Unlike prior works, we do not rely on a reference grasp, a coarse hand trajectory, or separate modes for grasping and articulating. To achieve this, we first generate distance-based contact maps conditioned on the object trajectory with an articulation-aware feature representation, revealing rich bimanual patterns for manipulation. The learned contact prior is then used to guide our hand motion generator, producing diverse and realistic bimanual motions for object movement and articulation. Our work offers key insights into feature representation and contact prior for articulated objects, demonstrating their effectiveness in taming the complex, high-dimensional space of bimanual hand-object interactions. Through comprehensive quantitative experiments, we demonstrate a clear step towards simplified and high-quality hand-object animations that surpass the state of the art in motion quality and diversity. Project page: https://vcai.mpi-inf.mpg.de/projects/bimart/.