Image fusion using symmetric skip autoencodervia an Adversarial Regulariser

TL;DR

This paper introduces a novel symmetric skip autoencoder with an adversarial regularizer for image fusion, effectively combining spatial details from visible images and spectral content from infrared images to produce realistic fused images.

Contribution

It presents a residual autoencoder architecture with symmetric skip connections and an adversarial regularizer, enhancing the quality of fused images by integrating spatial and spectral features.

Findings

Improved fused image realism and detail preservation

Effective integration of infrared spectral information

Enhanced inference on both visual and infrared images

Abstract

It is a challenging task to extract the best of both worlds by combining the spatial characteristics of a visible image and the spectral content of an infrared image. In this work, we propose a spatially constrained adversarial autoencoder that extracts deep features from the infrared and visible images to obtain a more exhaustive and global representation. In this paper, we propose a residual autoencoder architecture, regularised by a residual adversarial network, to generate a more realistic fused image. The residual module serves as primary building for the encoder, decoder and adversarial network, as an add on the symmetric skip connections perform the functionality of embedding the spatial characteristics directly from the initial layers of encoder structure to the decoder part of the network. The spectral information in the infrared image is incorporated by adding the feature maps over several layers in the encoder part of the fusion structure, which makes inference on both the visual and infrared images separately. In order to efficiently optimize the parameters of the network, we propose an adversarial regulariser network which would perform supervised learning on the fused image and the original visual image.