Depth-aware Blending of Smoothed Images for Bokeh Effect Generation

TL;DR

This paper presents a deep learning method that blends smoothed images with depth estimation to generate high-quality bokeh effects efficiently, suitable for smartphones and post-processing of images.

Contribution

It introduces a novel end-to-end framework that combines smoothed images and depth maps for realistic bokeh synthesis, outperforming existing methods in speed and quality.

Findings

Ranks second in AIM 2019 Bokeh Effect Challenge

Processes HD images in 0.03 seconds

Outperforms saliency-based baseline and previous approaches

Abstract

Bokeh effect is used in photography to capture images where the closer objects look sharp and every-thing else stays out-of-focus. Bokeh photos are generally captured using Single Lens Reflex cameras using shallow depth-of-field. Most of the modern smartphones can take bokeh images by leveraging dual rear cameras or a good auto-focus hardware. However, for smartphones with single-rear camera without a good auto-focus hardware, we have to rely on software to generate bokeh images. This kind of system is also useful to generate bokeh effect in already captured images. In this paper, an end-to-end deep learning framework is proposed to generate high-quality bokeh effect from images. The original image and different versions of smoothed images are blended to generate Bokeh effect with the help of a monocular depth estimation network. The proposed approach is compared against a saliency detection based baseline and a number of approaches proposed in AIM 2019 Challenge on Bokeh Effect Synthesis. Extensive experiments are shown in order to understand different parts of the proposed algorithm. The network is lightweight and can process an HD image in 0.03 seconds. This approach ranked second in AIM 2019 Bokeh effect challenge-Perceptual Track.