AutoRefiner: Improving Autoregressive Video Diffusion Models via Reflective Refinement Over the Stochastic Sampling Path

TL;DR

AutoRefiner introduces a novel noise refinement method tailored for autoregressive video diffusion models, significantly improving sample fidelity during stochastic sampling without increasing computational costs.

Contribution

The paper proposes AutoRefiner, a new noise refiner specifically designed for AR-VDMs, incorporating pathwise refinement and a reflective KV-cache to enhance sample quality efficiently.

Findings

AutoRefiner improves sample fidelity in AR-VDMs.

It operates as an efficient plug-in without extra training.

Enhanced video sample quality demonstrated in experiments.

Abstract

Autoregressive video diffusion models (AR-VDMs) show strong promise as scalable alternatives to bidirectional VDMs, enabling real-time and interactive applications. Yet there remains room for improvement in their sample fidelity. A promising solution is inference-time alignment, which optimizes the noise space to improve sample fidelity without updating model parameters. Yet, optimization- or search-based methods are computationally impractical for AR-VDMs. Recent text-to-image (T2I) works address this via feedforward noise refiners that modulate sampled noises in a single forward pass. Can such noise refiners be extended to AR-VDMs? We identify the failure of naively extending T2I noise refiners to AR-VDMs and propose AutoRefiner-a noise refiner tailored for AR-VDMs, with two key designs: pathwise noise refinement and a reflective KV-cache. Experiments demonstrate that AutoRefiner serves as an efficient plug-in for AR-VDMs, effectively enhancing sample fidelity by refining noise along stochastic denoising paths.