WarmPrior: Straightening Flow-Matching Policies with Temporal Priors

TL;DR

WarmPrior enhances diffusion-based robotic policies by using recent action history as a temporal prior, leading to straighter probability paths and improved success rates in manipulation and reinforcement learning.

Contribution

Introducing WarmPrior, a simple temporal prior that improves generative policies by shaping probability paths, with benefits in success rates and sample efficiency.

Findings

WarmPrior improves success rates in robotic manipulation tasks.

WarmPrior results in markedly straighter probability paths.

WarmPrior enhances exploration and final performance in reinforcement learning.

Abstract

Generative policies based on diffusion and flow matching have become a dominant paradigm for visuomotor robotic control. We show that replacing the standard Gaussian source distribution with WarmPrior, a simple temporally grounded prior constructed from readily available recent action history, consistently improves success rates on robotic manipulation tasks. We trace this gain to markedly straighter probability paths, echoing the effect of optimal-transport couplings in Rectified Flow. Beyond standard behavior cloning, WarmPrior also reshapes the exploration distribution in prior-space reinforcement learning, improving both sample efficiency and final performance. Collectively, these results identify the source distribution as an important and underexplored design axis in generative robot control.