Direct Handheld Burst Imaging to Simulated Defocus

TL;DR

This paper introduces a learning-based approach to generate shallow depth-of-field images from handheld burst photos, bypassing the need for explicit depth estimation and effectively handling challenging surfaces like reflections and transparency.

Contribution

The method synthesizes defocus blur directly from handheld bursts using deep learning, avoiding depth estimation errors common in traditional approaches.

Findings

Effective in handling reflective and transparent surfaces

Produces high-quality shallow depth-of-field images

Avoids artifacts from inaccurate depth estimation

Abstract

A shallow depth-of-field image keeps the subject in focus, and the foreground and background contexts blurred. This effect requires much larger lens apertures than those of smartphone cameras. Conventional methods acquire RGB-D images and blur image regions based on their depth. However, this approach is not suitable for reflective or transparent surfaces, or finely detailed object silhouettes, where the depth value is inaccurate or ambiguous. We present a learning-based method to synthesize the defocus blur in shallow depth-of-field images from handheld bursts acquired with a single small aperture lens. Our deep learning model directly produces the shallow depth-of-field image, avoiding explicit depth-based blurring. The simulated aperture diameter equals the camera translation during burst acquisition. Our method does not suffer from artifacts due to inaccurate or ambiguous depth estimation, and it is well-suited to portrait photography.