When to Lock Attention: Training-Free KV Control in Video Diffusion

TL;DR

KV-Lock is a training-free, adaptive framework for video diffusion models that dynamically balances background locking and foreground enhancement, reducing artifacts and improving video quality without additional training.

Contribution

We introduce KV-Lock, a novel training-free method that uses hallucination detection to adaptively control background locking and guidance in video diffusion models.

Findings

Outperforms existing methods in foreground quality and background fidelity

Effectively reduces artifacts in video editing tasks

Easily integrates with pre-trained DiT-based models

Abstract

Maintaining background consistency while enhancing foreground quality remains a core challenge in video editing. Injecting full-image information often leads to background artifacts, whereas rigid background locking severely constrains the model's capacity for foreground generation. To address this issue, we propose KV-Lock, a training-free framework tailored for DiT-based video diffusion models. Our core insight is that the hallucination metric (variance of denoising prediction) directly quantifies generation diversity, which is inherently linked to the classifier-free guidance (CFG) scale. Building upon this, KV-Lock leverages diffusion hallucination detection to dynamically schedule two key components: the fusion ratio between cached background key-values (KVs) and newly generated KVs, and the CFG scale. When hallucination risk is detected, KV-Lock strengthens background KV locking and simultaneously amplifies conditional guidance for foreground generation, thereby mitigating artifacts and improving generation fidelity. As a training-free, plug-and-play module, KV-Lock can be easily integrated into any pre-trained DiT-based models. Extensive experiments validate that our method outperforms existing approaches in improved foreground quality with high background fidelity across various video editing tasks.