Unsupervised Deep Multi-focus Image Fusion

TL;DR

This paper introduces an unsupervised deep learning method for multi-focus image fusion that directly predicts fully focused images without requiring synthetic training data, leveraging SSIM for loss calculation.

Contribution

It presents a novel CNN architecture trained without ground-truth fused images, enabling effective fusion using real benchmark datasets and SSIM-based loss.

Findings

Outperforms state-of-the-art methods in visual quality

Operates on images of variable sizes during testing

Achieves superior objective evaluation metrics

Abstract

Convolutional neural networks have recently been used for multi-focus image fusion. However, due to the lack of labeled data for supervised training of such networks, existing methods have resorted to adding Gaussian blur in focused images to simulate defocus and generate synthetic training data with ground-truth for supervised learning. Moreover, they classify pixels as focused or defocused and leverage the results to construct the fusion weight maps which then necessitates a series of post-processing steps. In this paper, we present unsupervised end-to-end learning for directly predicting the fully focused output image from multi-focus input image pairs. The proposed approach uses a novel CNN architecture trained to perform fusion without the need for ground truth fused images and exploits the image structural similarity (SSIM) to calculate the loss; a metric that is widely accepted for fused image quality evaluation. Consequently, we are able to utilize {\em real} benchmark datasets, instead of simulated ones, to train our network. The model is a feed-forward, fully convolutional neural network that can process images of variable sizes during test time. Extensive evaluations on benchmark datasets show that our method outperforms existing state-of-the-art in terms of visual quality and objective evaluations.