Image Super-Resolution Using T-Tetromino Pixels

TL;DR

This paper introduces a novel tetromino-shaped pixel binning approach combined with deep learning for improved single-image super-resolution, achieving higher image quality than traditional methods.

Contribution

It proposes a new tetromino pixel layout and demonstrates its effectiveness with a specialized neural network for superior super-resolution results.

Findings

Achieves up to +1.92 dB PSNR gain over conventional methods.

Uses a small repeating cell of four T-tetrominoes for sensor layout.

Demonstrates improved image quality in terms of PSNR and SSIM.

Abstract

For modern high-resolution imaging sensors, pixel binning is performed in low-lighting conditions and in case high frame rates are required. To recover the original spatial resolution, single-image super-resolution techniques can be applied for upscaling. To achieve a higher image quality after upscaling, we propose a novel binning concept using tetromino-shaped pixels. It is embedded into the field of compressed sensing and the coherence is calculated to motivate the sensor layouts used. Next, we investigate the reconstruction quality using tetromino pixels for the first time in literature. Instead of using different types of tetrominoes as proposed elsewhere, we show that using a small repeating cell consisting of only four T-tetrominoes is sufficient. For reconstruction, we use a locally fully connected reconstruction (LFCR) network as well as two classical reconstruction methods from the field of compressed sensing. Using the LFCR network in combination with the proposed tetromino layout, we achieve superior image quality in terms of PSNR, SSIM, and visually compared to conventional single-image super-resolution using the very deep super-resolution (VDSR) network. For PSNR, a gain of up to \SI[retain-explicit-plus]{+1.92}{dB} is achieved.