F2former: When Fractional Fourier Meets Deep Wiener Deconvolution and Selective Frequency Transformer for Image Deblurring

TL;DR

F2former introduces a novel image deblurring method combining Fractional Fourier Transform with transformer-based deep learning, effectively handling non-stationary signals and outperforming existing state-of-the-art techniques in motion and defocus deblurring.

Contribution

The paper proposes F2former, a new deep learning framework that integrates fractional Fourier transform with transformer architecture for improved non-stationary signal processing in image deblurring.

Findings

Outperforms state-of-the-art methods in motion deblurring.

Effective in defocus deblurring tasks.

Utilizes fractional frequency aware self-attention and frequency division multiplexing.

Abstract

Recent progress in image deblurring techniques focuses mainly on operating in both frequency and spatial domains using the Fourier transform (FT) properties. However, their performance is limited due to the dependency of FT on stationary signals and its lack of capability to extract spatial-frequency properties. In this paper, we propose a novel approach based on the Fractional Fourier Transform (FRFT), a unified spatial-frequency representation leveraging both spatial and frequency components simultaneously, making it ideal for processing non-stationary signals like images. Specifically, we introduce a Fractional Fourier Transformer (F2former), where we combine the classical fractional Fourier based Wiener deconvolution (F2WD) as well as a multi-branch encoder-decoder transformer based on a new fractional frequency aware transformer block (F2TB). We design F2TB consisting of a fractional frequency aware self-attention (F2SA) to estimate element-wise product attention based on important frequency components and a novel feed-forward network based on frequency division multiplexing (FM-FFN) to refine high and low frequency features separately for efficient latent clear image restoration. Experimental results for the cases of both motion deblurring as well as defocus deblurring show that the performance of our proposed method is superior to other state-of-the-art (SOTA) approaches.