DexFormer: Cross-Embodied Dexterous Manipulation via History-Conditioned Transformer

TL;DR

DexFormer is a transformer-based policy that uses historical observations to adapt to various dexterous hand embodiments, enabling zero-shot transfer and scalable manipulation across different robotic hands.

Contribution

Introduces DexFormer, a dynamics-aware, cross-embodiment policy that leverages temporal context to adapt to diverse robotic hand configurations in dexterous manipulation tasks.

Findings

Achieves strong zero-shot transfer to multiple robotic hands.

Generalizes manipulation skills across heterogeneous embodiments.

Establishes a scalable foundation for cross-embodiment dexterous manipulation.

Abstract

Dexterous manipulation remains one of the most challenging problems in robotics, requiring coherent control of high-DoF hands and arms under complex, contact-rich dynamics. A major barrier is embodiment variability: different dexterous hands exhibit distinct kinematics and dynamics, forcing prior methods to train separate policies or rely on shared action spaces with per-embodiment decoder heads. We present DexFormer, an end-to-end, dynamics-aware cross-embodiment policy built on a modified transformer backbone that conditions on historical observations. By using temporal context to infer morphology and dynamics on the fly, DexFormer adapts to diverse hand configurations and produces embodiment-appropriate control actions. Trained over a variety of procedurally generated dexterous-hand assets, DexFormer acquires a generalizable manipulation prior and exhibits strong zero-shot transfer to Leap Hand, Allegro Hand, and Rapid Hand. Our results show that a single policy can generalize across heterogeneous hand embodiments, establishing a scalable foundation for cross-embodiment dexterous manipulation. Project website: https://davidlxu.github.io/DexFormer-web/.