Generative Multi-Focus Image Fusion

TL;DR

This paper introduces GMFF, a two-stage generative framework for multi-focus image fusion that improves focus accuracy, restores details, and reduces artifacts using a combination of deterministic and generative models.

Contribution

The paper presents a novel two-stage generative fusion framework, combining StackMFF V4 and IFControlNet, to enhance multi-focus image fusion in complex real-world scenarios.

Findings

Achieves state-of-the-art fusion performance

Effectively restores missing focal content and details

Reduces edge artifacts in fused images

Abstract

Multi-focus image fusion aims to generate an all-in-focus image from a sequence of partially focused input images. Existing fusion algorithms generally assume that, for every spatial location in the scene, there is at least one input image in which that location is in focus. Furthermore, current fusion models often suffer from edge artifacts caused by uncertain focus estimation or hard-selection operations in complex real-world scenarios. To address these limitations, we propose a generative multi-focus image fusion framework, termed GMFF, which operates in two sequential stages. In the first stage, deterministic fusion is implemented using StackMFF V4, the latest version of the StackMFF series, and integrates the available focal plane information to produce an initial fused image. The second stage, generative restoration, is realized through IFControlNet, which leverages the generative capabilities of latent diffusion models to reconstruct content from missing focal planes, restore fine details, and eliminate edge artifacts. Each stage is independently developed and functions seamlessly in a cascaded manner. Extensive experiments demonstrate that GMFF achieves state-of-the-art fusion performance and exhibits significant potential for practical applications, particularly in scenarios involving complex multi-focal content. The implementation is publicly available at https://github.com/Xinzhe99/StackMFF-Series.