BokehOrNot: Transforming Bokeh Effect with Image Transformer and Lens Metadata Embedding

TL;DR

This paper introduces BokehOrNot, a universal model that embeds lens metadata and transforms bokeh effects in images, capable of handling both blur-to-sharp and sharp-to-blur effects across various lens and aperture combinations.

Contribution

It presents a novel method for embedding lens metadata into a model and a new loss calculation approach, enabling versatile bokeh effect transformation beyond existing models.

Findings

Outperforms current leading bokeh rendering models

Capable of both blur-to-sharp and sharp-to-blur transformations

Produces visually natural bokeh effects

Abstract

Bokeh effect is an optical phenomenon that offers a pleasant visual experience, typically generated by high-end cameras with wide aperture lenses. The task of bokeh effect transformation aims to produce a desired effect in one set of lenses and apertures based on another combination. Current models are limited in their ability to render a specific set of bokeh effects, primarily transformations from sharp to blur. In this paper, we propose a novel universal method for embedding lens metadata into the model and introducing a loss calculation method using alpha masks from the newly released Bokeh Effect Transformation Dataset(BETD) [3]. Based on the above techniques, we propose the BokehOrNot model, which is capable of producing both blur-to-sharp and sharp-to-blur bokeh effect with various combinations of lenses and aperture sizes. Our proposed model outperforms current leading bokeh rendering and image restoration models and renders visually natural bokeh effects. Our code is available at: https://github.com/indicator0/bokehornot.