Multi-FAN: Multi-Spectral Mosaic Super-Resolution Via Multi-Scale Feature Aggregation Network

TL;DR

This paper presents Multi-FAN, a multi-scale feature aggregation network for super-resolving multi-spectral mosaic images, improving image quality and inference speed by leveraging multi-scale features and super-resolution techniques.

Contribution

The paper introduces a novel Multi-FAN architecture that super-resolves mosaic images directly and enhances existing SR networks like RCAN with multi-scale feature aggregation.

Findings

Improved quantitative super-resolution results.

Enhanced inference time performance.

Effective integration with RCAN for better image quality.

Abstract

This paper introduces a novel method to super-resolve multi-spectral images captured by modern real-time single-shot mosaic image sensors, also known as multi-spectral cameras. Our contribution is two-fold. Firstly, we super-resolve multi-spectral images from mosaic images rather than image cubes, which helps to take into account the spatial offset of each wavelength. Secondly, we introduce an external multi-scale feature aggregation network (Multi-FAN) which concatenates the feature maps with different levels of semantic information throughout a super-resolution (SR) network. A cascade of convolutional layers then implicitly selects the most valuable feature maps to generate a mosaic image. This mosaic image is then merged with the mosaic image generated by the SR network to produce a quantitatively superior image. We apply our Multi-FAN to RCAN (Residual Channel Attention Network), which is the state-of-the-art SR algorithm. We show that Multi-FAN improves both quantitative results and well as inference time.