BGGAN: Bokeh-Glass Generative Adversarial Network for Rendering Realistic Bokeh

TL;DR

This paper introduces BGGAN, a novel GAN-based method with Glass-Net for generating realistic bokeh effects on smartphones without complex hardware, achieving high-quality rendering efficiently.

Contribution

The paper presents Glass-Net, a new generator architecture combined with perceptual loss and reimplemented Instance Normalization for fast, high-quality bokeh rendering on smartphones.

Findings

High-quality bokeh rendering on 1024x1536 images

Real-time processing in 1.9 seconds on smartphone GPUs

Achieved first place in AIM 2020 Bokeh Challenge

Abstract

A photo captured with bokeh effect often means objects in focus are sharp while the out-of-focus areas are all blurred. DSLR can easily render this kind of effect naturally. However, due to the limitation of sensors, smartphones cannot capture images with depth-of-field effects directly. In this paper, we propose a novel generator called Glass-Net, which generates bokeh images not relying on complex hardware. Meanwhile, the GAN-based method and perceptual loss are combined for rendering a realistic bokeh effect in the stage of finetuning the model. Moreover, Instance Normalization(IN) is reimplemented in our network, which ensures our tflite model with IN can be accelerated on smartphone GPU. Experiments show that our method is able to render a high-quality bokeh effect and process one $1024 \times 1536$ pixel image in 1.9 seconds on all smartphone chipsets. This approach ranked First in AIM 2020 Rendering Realistic Bokeh Challenge Track 1 \& Track 2.