Towards Photorealistic and Efficient Bokeh Rendering via Diffusion Framework

TL;DR

MagicBokeh is a diffusion-based framework that jointly enhances image resolution and renders photorealistic bokeh effects efficiently, especially on low-resolution images from mobile devices.

Contribution

It introduces a unified diffusion model with a focus-aware attention and degradation-aware depth module for improved bokeh rendering and super-resolution.

Findings

Produces photorealistic bokeh effects on low-resolution images

Jointly optimizes bokeh rendering and super-resolution

Achieves high visual fidelity and controllability

Abstract

Existing mobile devices are constrained by compact optical designs, such as small apertures, which make it difficult to produce natural, optically realistic bokeh effects. Although recent learning-based methods have shown promising results, they still struggle with photos captured under high digital zoom levels, which often suffer from reduced resolution and loss of fine details. A naive solution is to enhance image quality before applying bokeh rendering, yet this two-stage pipeline reduces efficiency and introduces unnecessary error accumulation. To overcome these limitations, we propose MagicBokeh, a unified diffusion-based framework designed for high-quality and efficient bokeh rendering. Through an alternative training strategy and a focus-aware masked attention mechanism, our method jointly optimizes bokeh rendering and super-resolution, substantially improving both controllability and visual fidelity. Furthermore, we introduce degradation-aware depth module to enable more accurate depth estimation from low-quality inputs. Experimental results demonstrate that MagicBokeh efficiently produces photorealistic bokeh effects, particularly on real-world low-resolution images, paving the way for future advancements in bokeh rendering. Our code and models are available at https://github.com/vivoCameraResearch/MagicBokeh.