Primary-Fine Decoupling for Action Generation in Robotic Imitation

TL;DR

This paper introduces PF-DAG, a two-stage framework that improves robotic action generation by decoupling coarse mode selection from fine-grained continuous action synthesis, leading to more stable and accurate imitation learning.

Contribution

The paper proposes a novel two-stage decoupling approach for multi-modal action generation in robotics, combining discrete mode selection with continuous action refinement, and provides theoretical and empirical validation.

Findings

PF-DAG achieves lower MSE bounds than single-stage policies.

Outperforms state-of-the-art methods across 56 manipulation tasks.

Successfully generalizes to real-world tactile dexterous manipulation.

Abstract

Multi-modal distribution in robotic manipulation action sequences poses critical challenges for imitation learning. To this end, existing approaches often model the action space as either a discrete set of tokens or a continuous, latent-variable distribution. However, both approaches present trade-offs: some methods discretize actions into tokens and therefore lose fine-grained action variations, while others generate continuous actions in a single stage tend to produce unstable mode transitions. To address these limitations, we propose Primary-Fine Decoupling for Action Generation (PF-DAG), a two-stage framework that decouples coarse action consistency from fine-grained variations. First, we compress action chunks into a small set of discrete modes, enabling a lightweight policy to select consistent coarse modes and avoid mode bouncing. Second, a mode conditioned MeanFlow policy is learned to generate high-fidelity continuous actions. Theoretically, we prove PF-DAG's two-stage design achieves a strictly lower MSE bound than single-stage generative policies. Empirically, PF-DAG outperforms state-of-the-art baselines across 56 tasks from Adroit, DexArt, and MetaWorld benchmarks. It further generalizes to real-world tactile dexterous manipulation tasks. Our work demonstrates that explicit mode-level decoupling enables both robust multi-modal modeling and reactive closed-loop control for robotic manipulation.