Bokeh Rendering Based on Adaptive Depth Calibration Network

TL;DR

This paper introduces a novel deep learning approach using a Vision Transformer and adaptive depth calibration to improve bokeh rendering quality in mobile photography, addressing hardware limitations of mobile devices.

Contribution

The paper proposes an adaptive depth calibration network with Vision Transformer for high-quality bokeh rendering, outperforming existing methods in accuracy and visual quality.

Findings

Achieves 24.7% improvement on LPIPS metric.

Outperforms state-of-the-art methods in PSNR scores.

Generates high-resolution, realistic bokeh images.

Abstract

Bokeh rendering is a popular and effective technique used in photography to create an aesthetically pleasing effect. It is widely used to blur the background and highlight the subject in the foreground, thereby drawing the viewer's attention to the main focus of the image. In traditional digital single-lens reflex cameras (DSLRs), this effect is achieved through the use of a large aperture lens. This allows the camera to capture images with shallow depth-of-field, in which only a small area of the image is in sharp focus, while the rest of the image is blurred. However, the hardware embedded in mobile phones is typically much smaller and more limited than that found in DSLRs. Consequently, mobile phones are not able to capture natural shallow depth-of-field photos, which can be a significant limitation for mobile photography. To address this challenge, in this paper, we propose a novel method for bokeh rendering using the Vision Transformer, a recent and powerful deep learning architecture. Our approach employs an adaptive depth calibration network that acts as a confidence level to compensate for errors in monocular depth estimation. This network is used to supervise the rendering process in conjunction with depth information, allowing for the generation of high-quality bokeh images at high resolutions. Our experiments demonstrate that our proposed method outperforms state-of-the-art methods, achieving about 24.7% improvements on LPIPS and obtaining higher PSNR scores.