DiffFNO: Diffusion Fourier Neural Operator

TL;DR

DiffFNO introduces a diffusion-based super-resolution framework combining spectral and spatial features, achieving state-of-the-art results with improved accuracy and efficiency across various scaling factors.

Contribution

The paper presents DiffFNO, a novel diffusion framework with a Weighted Fourier Neural Operator and adaptive sampling, advancing super-resolution performance and speed.

Findings

Outperforms existing methods by 2-4 dB PSNR across scales

Achieves superior accuracy with lower inference time

Effective high-frequency detail reconstruction

Abstract

We introduce DiffFNO, a novel diffusion framework for arbitrary-scale super-resolution strengthened by a Weighted Fourier Neural Operator (WFNO). Mode Rebalancing in WFNO effectively captures critical frequency components, significantly improving the reconstruction of high-frequency image details that are crucial for super-resolution tasks. Gated Fusion Mechanism (GFM) adaptively complements WFNO's spectral features with spatial features from an Attention-based Neural Operator (AttnNO). This enhances the network's capability to capture both global structures and local details. Adaptive Time-Step (ATS) ODE solver, a deterministic sampling strategy, accelerates inference without sacrificing output quality by dynamically adjusting integration step sizes ATS. Extensive experiments demonstrate that DiffFNO achieves state-of-the-art (SOTA) results, outperforming existing methods across various scaling factors by a margin of 2-4 dB in PSNR, including those beyond the training distribution. It also achieves this at lower inference time. Our approach sets a new standard in super-resolution, delivering both superior accuracy and computational efficiency.