RoTri-Diff: A Spatial Robot-Object Triadic Interaction-Guided Diffusion Model for Bimanual Manipulation

TL;DR

RoTri-Diff introduces a novel diffusion-based imitation learning framework that explicitly models the spatial triadic relationship among robot arms and objects, improving coordination and stability in bimanual manipulation tasks.

Contribution

It proposes the RoTri representation for modeling spatial relationships and integrates it into a diffusion framework for enhanced bimanual manipulation.

Findings

Outperforms state-of-the-art by 10.2% on RLBench2 tasks

Achieves stable performance on 4 real-world bimanual tasks

Effectively captures spatial relationships to reduce collisions and improve stability

Abstract

Bimanual manipulation is a fundamental robotic skill that requires continuous and precise coordination between two arms. While imitation learning (IL) is the dominant paradigm for acquiring this capability, existing approaches, whether robot-centric or object-centric, often overlook the dynamic geometric relationship among the two arms and the manipulated object. This limitation frequently leads to inter-arm collisions, unstable grasps, and degraded performance in complex tasks. To address this, in this paper we explicitly models the Robot-Object Triadic Interaction (RoTri) representation in bimanual systems, by encoding the relative 6D poses between the two arms and the object to capture their spatial triadic relationship and establish continuous triangular geometric constraints. Building on this, we further introduce RoTri-Diff, a diffusion-based imitation learning framework that combines RoTri constraints with robot keyposes and object motion in a hierarchical diffusion process. This enables the generation of stable, coordinated trajectories and robust execution across different modes of bimanual manipulation. Extensive experiments show that our approach outperforms state-of-the-art baselines by 10.2% on 11 representative RLBench2 tasks and achieves stable performance on 4 challenging real-world bimanual tasks. Project website: https://rotri-diff.github.io/.