Realistic Bokeh Effect Rendering on Mobile GPUs, Mobile AI & AIM 2022 challenge: Report

TL;DR

This paper reports on a challenge to develop efficient AI-based bokeh rendering models for mobile GPUs, utilizing a large dataset and focusing on real-time performance on smartphones.

Contribution

It introduces a new challenge and dataset for mobile bokeh rendering, and provides detailed descriptions of models optimized for TensorFlow Lite on smartphone GPUs.

Findings

Models achieved real-time performance on Kirin 9000's Mali GPU

Large-scale dataset of 5K image pairs provided for training and evaluation

Detailed analysis of models' efficiency and quality on mobile hardware

Abstract

As mobile cameras with compact optics are unable to produce a strong bokeh effect, lots of interest is now devoted to deep learning-based solutions for this task. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based bokeh effect rendering approach that can run on modern smartphone GPUs using TensorFlow Lite. The participants were provided with a large-scale EBB! bokeh dataset consisting of 5K shallow / wide depth-of-field image pairs captured using the Canon 7D DSLR camera. The runtime of the resulting models was evaluated on the Kirin 9000's Mali GPU that provides excellent acceleration results for the majority of common deep learning ops. A detailed description of all models developed in this challenge is provided in this paper.