DAE-Fuse: An Adaptive Discriminative Autoencoder for Multi-Modality Image Fusion

TL;DR

DAE-Fuse introduces an adaptive autoencoder framework that produces sharp, natural multi-modality image fusion results, extending to video with temporal consistency, and outperforms existing methods on various benchmarks.

Contribution

It presents a novel two-phase discriminative autoencoder for improved image fusion, extending techniques to video with temporal consistency, and demonstrates superior performance on multiple datasets.

Findings

Achieves state-of-the-art fusion quality on benchmarks.

Extends image fusion to video with temporal consistency.

Demonstrates superior generalizability to medical imaging.

Abstract

In extreme scenarios such as nighttime or low-visibility environments, achieving reliable perception is critical for applications like autonomous driving, robotics, and surveillance. Multi-modality image fusion, particularly integrating infrared imaging, offers a robust solution by combining complementary information from different modalities to enhance scene understanding and decision-making. However, current methods face significant limitations: GAN-based approaches often produce blurry images that lack fine-grained details, while AE-based methods may introduce bias toward specific modalities, leading to unnatural fusion results. To address these challenges, we propose DAE-Fuse, a novel two-phase discriminative autoencoder framework that generates sharp and natural fused images. Furthermore, We pioneer the extension of image fusion techniques from static images to the video domain while preserving temporal consistency across frames, thus advancing the perceptual capabilities required for autonomous navigation. Extensive experiments on public datasets demonstrate that DAE-Fuse achieves state-of-the-art performance on multiple benchmarks, with superior generalizability to tasks like medical image fusion.