BokehDiff: Neural Lens Blur with One-Step Diffusion

TL;DR

BokehDiff is a novel lens blur rendering technique that uses diffusion models and physics-inspired modules to produce accurate, artifact-free images with realistic depth effects, even with limited training data.

Contribution

It introduces a physics-inspired self-attention module and a one-step diffusion inference scheme for high-quality, artifact-free lens blur rendering.

Findings

Achieves physically accurate and visually appealing lens blur effects.

Effectively handles depth discontinuities without artifacts.

Synthesizes photorealistic foregrounds with diffusion models.

Abstract

We introduce BokehDiff, a novel lens blur rendering method that achieves physically accurate and visually appealing outcomes, with the help of generative diffusion prior. Previous methods are bounded by the accuracy of depth estimation, generating artifacts in depth discontinuities. Our method employs a physics-inspired self-attention module that aligns with the image formation process, incorporating depth-dependent circle of confusion constraint and self-occlusion effects. We adapt the diffusion model to the one-step inference scheme without introducing additional noise, and achieve results of high quality and fidelity. To address the lack of scalable paired data, we propose to synthesize photorealistic foregrounds with transparency with diffusion models, balancing authenticity and scene diversity.