Adaptive Diffusion Constrained Sampling for Bimanual Robot Manipulation

TL;DR

This paper introduces ADCS, a novel generative framework that integrates multiple geometric constraints into a diffusion model for improved bimanual robot manipulation, enhancing flexibility and precision.

Contribution

The paper presents a Transformer-based adaptive weighting mechanism and a two-phase sampling strategy for constraint-aware diffusion modeling in robot manipulation.

Findings

Enhanced sample diversity and generalization in dual-arm tasks

Effective integration of equality and inequality constraints

Improved precision in complex manipulation scenarios

Abstract

Coordinated multi-arm manipulation requires satisfying multiple simultaneous geometric constraints across high-dimensional configuration spaces, which poses a significant challenge for traditional planning and control methods. In this work, we propose Adaptive Diffusion Constrained Sampling (ADCS), a generative framework that flexibly integrates both equality (e.g., relative and absolute pose constraints) and structured inequality constraints (e.g., proximity to object surfaces) into an energy-based diffusion model. Equality constraints are modeled using dedicated energy networks trained on pose differences in Lie algebra space, while inequality constraints are represented via Signed Distance Functions (SDFs) and encoded into learned constraint embeddings, allowing the model to reason about complex spatial regions. A key innovation of our method is a Transformer-based architecture that learns to weight constraint-specific energy functions at inference time, enabling flexible and context-aware constraint integration. Moreover, we adopt a two-phase sampling strategy that improves precision and sample diversity by combining Langevin dynamics with resampling and density-aware re-weighting. Experimental results on dual-arm manipulation tasks show that ADCS significantly improves sample diversity and generalization across settings demanding precise coordination and adaptive constraint handling.