A Novel End-To-End Network for Reconstruction of Non-Regularly Sampled Image Data Using Locally Fully Connected Layers

TL;DR

This paper introduces an end-to-end neural network combining locally fully connected layers and VDSR to reconstruct high-resolution images from non-regularly sampled sensor data, significantly improving image quality.

Contribution

The paper presents a novel neural network architecture that effectively reconstructs high-resolution images from non-regular sensor sampling, outperforming existing methods.

Findings

PSNR increased by 2.96 dB on Urban100 dataset

Achieved 1.11 dB gain over low-res sensor with VDSR

Demonstrated effectiveness of locally fully connected layers in reconstruction

Abstract

Quarter sampling and three-quarter sampling are novel sensor concepts that enable the acquisition of higher resolution images without increasing the number of pixels. This is achieved by non-regularly covering parts of each pixel of a low-resolution sensor such that only one quadrant or three quadrants of the sensor area of each pixel is sensitive to light. Combining a properly designed mask and a high-quality reconstruction algorithm, a higher image quality can be achieved than using a low-resolution sensor and subsequent upsampling. For the latter case, the image quality can be further enhanced using super resolution algorithms such as the very deep super resolution network (VDSR). In this paper, we propose a novel end-to-end neural network to reconstruct high resolution images from non-regularly sampled sensor data. The network is a concatenation of a locally fully connected reconstruction network (LFCR) and a standard VDSR network. Altogether, using a three-quarter sampling sensor with our novel neural network layout, the image quality in terms of PSNR for the Urban100 dataset can be increased by 2.96 dB compared to the state-of-the-art approach. Compared to a low-resolution sensor with VDSR, a gain of 1.11 dB is achieved.