Learning to Deblur using Light Field Generated and Real Defocus Images

TL;DR

This paper introduces a deep learning method for defocus deblurring that combines light field-generated data with real captured images, achieving state-of-the-art results by addressing data accuracy issues.

Contribution

The proposed approach leverages light field data for initial training and fine-tunes with real images to improve defocus deblurring performance, overcoming domain differences.

Findings

Achieves state-of-the-art quantitative results.

Demonstrates effective domain adaptation from synthetic to real images.

Provides comprehensive ablation studies on network modules.

Abstract

Defocus deblurring is a challenging task due to the spatially varying nature of defocus blur. While deep learning approach shows great promise in solving image restoration problems, defocus deblurring demands accurate training data that consists of all-in-focus and defocus image pairs, which is difficult to collect. Naive two-shot capturing cannot achieve pixel-wise correspondence between the defocused and all-in-focus image pairs. Synthetic aperture of light fields is suggested to be a more reliable way to generate accurate image pairs. However, the defocus blur generated from light field data is different from that of the images captured with a traditional digital camera. In this paper, we propose a novel deep defocus deblurring network that leverages the strength and overcomes the shortcoming of light fields. We first train the network on a light field-generated dataset for its highly accurate image correspondence. Then, we fine-tune the network using feature loss on another dataset collected by the two-shot method to alleviate the differences between the defocus blur exists in the two domains. This strategy is proved to be highly effective and able to achieve the state-of-the-art performance both quantitatively and qualitatively on multiple test sets. Extensive ablation studies have been conducted to analyze the effect of each network module to the final performance.