Streaming Diffusion Policy: Fast Policy Synthesis with Variable Noise Diffusion Models

TL;DR

The paper introduces Streaming Diffusion Policy (SDP), a fast policy synthesis method that generates partially denoised action trajectories with variable noise levels, enabling rapid and accurate robotic decision-making.

Contribution

SDP offers a computationally efficient alternative to diffusion process distillation by producing partially denoised actions, significantly speeding up policy synthesis without sacrificing accuracy.

Findings

Dramatically speeds up policy synthesis in robotic tasks.

Maintains performance in both simulated and real-world environments.

Outperforms existing diffusion distillation methods in efficiency.

Abstract

Diffusion models have seen rapid adoption in robotic imitation learning, enabling autonomous execution of complex dexterous tasks. However, action synthesis is often slow, requiring many steps of iterative denoising, limiting the extent to which models can be used in tasks that require fast reactive policies. To sidestep this, recent works have explored how the distillation of the diffusion process can be used to accelerate policy synthesis. However, distillation is computationally expensive and can hurt both the accuracy and diversity of synthesized actions. We propose SDP (Streaming Diffusion Policy), an alternative method to accelerate policy synthesis, leveraging the insight that generating a partially denoised action trajectory is substantially faster than a full output action trajectory. At each observation, our approach outputs a partially denoised action trajectory with variable levels of noise corruption, where the immediate action to execute is noise-free, with subsequent actions having increasing levels of noise and uncertainty. The partially denoised action trajectory for a new observation can then be quickly generated by applying a few steps of denoising to the previously predicted noisy action trajectory (rolled over by one timestep). We illustrate the efficacy of this approach, dramatically speeding up policy synthesis while preserving performance across both simulated and real-world settings.