Inter-Image Pixel Shuffling for Multi-focus Image Fusion

TL;DR

This paper introduces Inter-image Pixel Shuffling (IPS), a novel training method enabling neural networks to perform multi-focus image fusion without needing actual multi-focus training data, by reformulating the task as pixel-wise classification.

Contribution

The paper proposes IPS, a new data augmentation technique that generates training data for multi-focus image fusion from single images, improving fusion performance without multi-focus training datasets.

Findings

IPS outperforms existing methods in quality

Effective training without multi-focus images

Combines CNNs with state space models for better results

Abstract

Multi-focus image fusion aims to combine multiple partially focused images into a single all-in-focus image. Although deep learning has shown promise in this task, its effectiveness is often limited by the scarcity of suitable training data. This paper introduces Inter-image Pixel Shuffling (IPS), a novel method that allows neural networks to learn multi-focus image fusion without requiring actual multi-focus images. IPS reformulates the task as a pixel-wise classification problem, where the goal is to identify the focused pixel from a pixel group at each spatial position. In this method, pixels from a clear optical image are treated as focused, while pixels from a low-pass filtered version of the same image are considered defocused. By randomly shuffling the focused and defocused pixels at identical spatial positions in the original and filtered images, IPS generates training data that preserves spatial structure while mixing focus-defocus information. The model is trained to select the focused pixel from each spatially aligned pixel group, thus learning to reconstruct an all-in-focus image by aggregating sharp content from the input. To further enhance fusion quality, IPS adopts a cross-image fusion network that integrates the localized representation power of convolutional neural networks with the long-range modeling capabilities of state space models. This design effectively leverages both spatial detail and contextual information to produce high-quality fused results. Experimental results indicate that IPS significantly outperforms existing multi-focus image fusion methods, even without training on multi-focus images.