Learning Generalizable Visuomotor Policy through Dynamics-Alignment

TL;DR

This paper introduces DAP, a novel policy learning method that integrates dynamics prediction with mutual feedback, significantly improving robot manipulation generalization, especially in out-of-distribution scenarios with visual distractions.

Contribution

The paper presents a new architecture combining dynamics prediction with policy learning, enabling self-correction and better generalization in robotic manipulation tasks.

Findings

Superior generalization on real-world tasks

Robustness to visual distractions and lighting changes

Outperforms baseline methods in OOD scenarios

Abstract

Behavior cloning methods for robot learning suffer from poor generalization due to limited data support beyond expert demonstrations. Recent approaches leveraging video prediction models have shown promising results by learning rich spatiotemporal representations from large-scale datasets. However, these models learn action-agnostic dynamics that cannot distinguish between different control inputs, limiting their utility for precise manipulation tasks and requiring large pretraining datasets. We propose a Dynamics-Aligned Flow Matching Policy (DAP) that integrates dynamics prediction into policy learning. Our method introduces a novel architecture where policy and dynamics models provide mutual corrective feedback during action generation, enabling self-correction and improved generalization. Empirical validation demonstrates generalization performance superior to baseline methods on real-world robotic manipulation tasks, showing particular robustness in OOD scenarios including visual distractions and lighting variations.