Rendering Natural Camera Bokeh Effect with Deep Learning

TL;DR

This paper introduces a deep learning approach trained on a large dataset of DSLR images to generate realistic bokeh effects on mobile camera photos, surpassing traditional Gaussian blur methods.

Contribution

It presents a novel large-scale dataset and a deep learning model for realistic shallow depth-of-field rendering from single images.

Findings

The model produces plausible non-uniform bokeh effects.

It outperforms Gaussian blur-based methods.

The dataset and code are publicly available.

Abstract

Bokeh is an important artistic effect used to highlight the main object of interest on the photo by blurring all out-of-focus areas. While DSLR and system camera lenses can render this effect naturally, mobile cameras are unable to produce shallow depth-of-field photos due to a very small aperture diameter of their optics. Unlike the current solutions simulating bokeh by applying Gaussian blur to image background, in this paper we propose to learn a realistic shallow focus technique directly from the photos produced by DSLR cameras. For this, we present a large-scale bokeh dataset consisting of 5K shallow / wide depth-of-field image pairs captured using the Canon 7D DSLR with 50mm f/1.8 lenses. We use these images to train a deep learning model to reproduce a natural bokeh effect based on a single narrow-aperture image. The experimental results show that the proposed approach is able to render a plausible non-uniform bokeh even in case of complex input data with multiple objects. The dataset, pre-trained models and codes used in this paper are available on the project website.