Improving Single-Image Defocus Deblurring: How Dual-Pixel Images Help Through Multi-Task Learning

TL;DR

This paper introduces a multi-task learning approach that leverages dual-pixel data to improve single-image defocus deblurring, achieving higher PSNR and enabling accurate DP-view synthesis for enhanced image restoration.

Contribution

The authors propose a novel multi-task network that jointly deblurs images and synthesizes dual-pixel views, significantly improving deblurring performance and enabling new applications.

Findings

+1dB PSNR improvement over state-of-the-art methods

High-quality DP-view synthesis (~39dB PSNR) from a single image

Captured a new dataset of 7,059 images for training and evaluation

Abstract

Many camera sensors use a dual-pixel (DP) design that operates as a rudimentary light field providing two sub-aperture views of a scene in a single capture. The DP sensor was developed to improve how cameras perform autofocus. Since the DP sensor's introduction, researchers have found additional uses for the DP data, such as depth estimation, reflection removal, and defocus deblurring. We are interested in the latter task of defocus deblurring. In particular, we propose a single-image deblurring network that incorporates the two sub-aperture views into a multi-task framework. Specifically, we show that jointly learning to predict the two DP views from a single blurry input image improves the network's ability to learn to deblur the image. Our experiments show this multi-task strategy achieves +1dB PSNR improvement over state-of-the-art defocus deblurring methods. In addition, our multi-task framework allows accurate DP-view synthesis (e.g., ~39dB PSNR) from the single input image. These high-quality DP views can be used for other DP-based applications, such as reflection removal. As part of this effort, we have captured a new dataset of 7,059 high-quality images to support our training for the DP-view synthesis task. Our dataset, code, and trained models are publicly available at https://github.com/Abdullah-Abuolaim/multi-task-defocus-deblurring-dual-pixel-nimat.