Data Acquisition and Preparation for Dual-reference Deep Learning of Image Super-Resolution

TL;DR

This paper introduces a practical data collection method using real camera captures and a dual-domain registration technique to improve deep learning-based image super-resolution, resulting in higher quality models tailored to specific cameras.

Contribution

It proposes a novel camera-based data acquisition process and registration method to generate more accurate training pairs for super-resolution models, addressing limitations of synthetic data.

Findings

Supervised models trained on the new dataset outperform those trained on synthetic data.

The screen-capturing process is automated, low-cost, and adaptable to different cameras.

High sub-pixel alignment accuracy enhances super-resolution training quality.

Abstract

The performance of deep learning based image super-resolution (SR) methods depend on how accurately the paired low and high resolution images for training characterize the sampling process of real cameras. Low and high resolution (LR$\sim$HR) image pairs synthesized by degradation models (e.g., bicubic downsampling) deviate from those in reality; thus the synthetically-trained DCNN SR models work disappointingly when being applied to real-world images. To address this issue, we propose a novel data acquisition process to shoot a large set of LR$\sim$HR image pairs using real cameras. The images are displayed on an ultra-high quality screen and captured at different resolutions. The resulting LR$\sim$HR image pairs can be aligned at very high sub-pixel precision by a novel spatial-frequency dual-domain registration method, and hence they provide more appropriate training data for the learning task of super-resolution. Moreover, the captured HR image and the original digital image offer dual references to strengthen supervised learning. Experimental results show that training a super-resolution DCNN by our LR$\sim$HR dataset achieves higher image quality than training it by other datasets in the literature. Moreover, the proposed screen-capturing data collection process can be automated; it can be carried out for any target camera with ease and low cost, offering a practical way of tailoring the training of a DCNN SR model separately to each of the given cameras.