Any-to-Bokeh: Arbitrary-Subject Video Refocusing with Video Diffusion Model

TL;DR

This paper introduces a novel diffusion-based method for generating temporally coherent, depth-aware video bokeh effects, addressing previous issues of flickering and lack of control, and establishing a new benchmark in controllable video depth-of-field rendering.

Contribution

It presents the first dedicated diffusion framework for video bokeh, utilizing multi-plane image conditioning and progressive training for improved stability and controllability.

Findings

Outperforms prior methods in temporal coherence and spatial accuracy.

Demonstrates effectiveness on synthetic and real-world benchmarks.

Provides controllable depth-of-field effects with enhanced stability.

Abstract

Diffusion models have recently emerged as powerful tools for camera simulation, enabling both geometric transformations and realistic optical effects. Among these, image-based bokeh rendering has shown promising results, but diffusion for video bokeh remains unexplored. Existing image-based methods are plagued by temporal flickering and inconsistent blur transitions, while current video editing methods lack explicit control over the focus plane and bokeh intensity. These issues limit their applicability for controllable video bokeh. In this work, we propose a one-step diffusion framework for generating temporally coherent, depth-aware video bokeh rendering. The framework employs a multi-plane image (MPI) representation adapted to the focal plane to condition the video diffusion model, thereby enabling it to exploit strong 3D priors from pretrained backbones. To further enhance temporal stability, depth robustness, and detail preservation, we introduce a progressive training strategy. Experiments on synthetic and real-world benchmarks demonstrate superior temporal coherence, spatial accuracy, and controllability, outperforming prior baselines. This work represents the first dedicated diffusion framework for video bokeh generation, establishing a new baseline for temporally coherent and controllable depth-of-field effects.