Learning to Refocus with Video Diffusion Models

TL;DR

This paper presents a novel method using video diffusion models to generate realistic, interactive refocusing from a single defocused image, enabling post-capture focus adjustments and enhancing photography editing capabilities.

Contribution

It introduces a new approach for post-capture refocusing with video diffusion models and provides a large-scale dataset for future research.

Findings

Outperforms existing methods in perceptual quality

Robust across diverse real-world scenarios

Enables interactive focus editing

Abstract

Focus is a cornerstone of photography, yet autofocus systems often fail to capture the intended subject, and users frequently wish to adjust focus after capture. We introduce a novel method for realistic post-capture refocusing using video diffusion models. From a single defocused image, our approach generates a perceptually accurate focal stack, represented as a video sequence, enabling interactive refocusing and unlocking a range of downstream applications. We release a large-scale focal stack dataset acquired under diverse real-world smartphone conditions to support this work and future research. Our method consistently outperforms existing approaches in both perceptual quality and robustness across challenging scenarios, paving the way for more advanced focus-editing capabilities in everyday photography. Code and data are available at www.learn2refocus.github.io