BAgger: Backwards Aggregation for Mitigating Drift in Autoregressive Video Diffusion Models

TL;DR

BAgger is a novel self-supervised training scheme for autoregressive video models that reduces drift and improves long-term stability by teaching models to recover from their own mistakes without relying on distillation.

Contribution

Introduces BAgger, a new training method that mitigates drift in autoregressive video models using self-supervised correction trajectories, avoiding complex distillation.

Findings

More stable long-horizon motion in generated videos

Improved visual consistency and reduced drift

Effective on text-to-video and multi-prompt generation tasks

Abstract

Autoregressive video models are promising for world modeling via next-frame prediction, but they suffer from exposure bias: a mismatch between training on clean contexts and inference on self-generated frames, causing errors to compound and quality to drift over time. We introduce Backwards Aggregation (BAgger), a self-supervised scheme that constructs corrective trajectories from the model's own rollouts, teaching it to recover from its mistakes. Unlike prior approaches that rely on few-step distillation and distribution-matching losses, which can hurt quality and diversity, BAgger trains with standard score or flow matching objectives, avoiding large teachers and long-chain backpropagation through time. We instantiate BAgger on causal diffusion transformers and evaluate on text-to-video, video extension, and multi-prompt generation, observing more stable long-horizon motion and better visual consistency with reduced drift.