Super-resolved Chromatic Mapping of Snapshot Mosaic Image Sensors via a Texture Sensitive Residual Network

TL;DR

This paper presents a novel texture-sensitive residual network that super-resolves and accurately maps colors in images from snapshot mosaic sensors, overcoming resolution and chromatic mapping challenges with state-of-the-art performance.

Contribution

Introduces a residual channel attention network with texture-sensitive blocks for improved super-resolution and chromatic mapping of snapshot mosaic sensor images, surpassing traditional methods.

Findings

Outperforms traditional interpolation and color matching approaches

Establishes state-of-the-art results in chromatic mapping

Provides a new dataset of 296 stereo multi-spectral/RGB image pairs

Abstract

This paper introduces a novel method to simultaneously super-resolve and colour-predict images acquired by snapshot mosaic sensors. These sensors allow for spectral images to be acquired using low-power, small form factor, solid-state CMOS sensors that can operate at video frame rates without the need for complex optical setups. Despite their desirable traits, their main drawback stems from the fact that the spatial resolution of the imagery acquired by these sensors is low. Moreover, chromatic mapping in snapshot mosaic sensors is not straightforward since the bands delivered by the sensor tend to be narrow and unevenly distributed across the range in which they operate. We tackle this drawback as applied to chromatic mapping by using a residual channel attention network equipped with a texture sensitive block. Our method significantly outperforms the traditional approach of interpolating the image and, afterwards, applying a colour matching function. This work establishes state-of-the-art in this domain while also making available to the research community a dataset containing 296 registered stereo multi-spectral/RGB images pairs.