Relax Forcing: Relaxed KV-Memory for Consistent Long Video Generation

TL;DR

This paper introduces Relax Forcing, a structured temporal memory mechanism for autoregressive diffusion models, which improves long video generation by effectively utilizing relevant past information to enhance motion dynamics and temporal consistency.

Contribution

The paper proposes Relax Forcing, a novel structured memory approach that decomposes temporal context into functional roles, significantly improving long-horizon video generation quality.

Findings

Enhances motion dynamics and temporal consistency in long videos.

Reduces attention overhead during inference.

Structured memory outperforms dense history in long video synthesis.

Abstract

Autoregressive (AR) video diffusion has recently emerged as a promising paradigm for long video generation, enabling causal synthesis beyond the limits of bidirectional models. To address training-inference mismatch, a series of self-forcing strategies have been proposed to improve rollout stability by conditioning the model on its own predictions during training. While these approaches substantially mitigate exposure bias, extending generation to minute-scale horizons remains challenging due to progressive temporal degradation. In this work, we show that this limitation is not primarily caused by insufficient memory, but by how temporal memory is utilised during inference. Through empirical analysis, we find that increasing memory does not consistently improve long-horizon generation, and that the temporal placement of historical context significantly influences motion dynamics while leaving visual quality largely unchanged. These findings suggest that temporal memory should not be treated as a homogeneous buffer. Motivated by this insight, we introduce Relax Forcing, a structured temporal memory mechanism for AR diffusion. Instead of attending to the dense generated history, Relax Forcing decomposes temporal context into three functional roles: Sink for global stability, Tail for short-term continuity, and dynamically selected History for structural motion guidance, and selectively incorporates only the most relevant past information. This design mitigates error accumulation during extrapolation while preserving motion evolution. Experiments on VBench-Long demonstrate that Relax Forcing improves motion dynamics and overall temporal consistency while reducing attention overhead. Our results suggest that structured temporal memory is essential for scalable long video generation, complementing existing forcing-based training strategies.