Exploring Richer and More Accurate Information via Frequency Selection for Image Restoration

TL;DR

This paper introduces MSFSNet, a multi-scale frequency selection network that combines spatial and frequency domain knowledge with novel modules to improve image restoration accuracy and detail recovery.

Contribution

The paper proposes generic plug-in modules, DFS and SFF, that enhance existing networks by integrating frequency selection and multi-scale feature fusion for better image restoration.

Findings

MSFSNet outperforms or matches state-of-the-art methods across various tasks.

DFS and SFF modules improve feature discrimination and information propagation.

The approach effectively captures rich frequency information for clearer images.

Abstract

Image restoration aims to recover high-quality images from their corrupted counterparts. Many existing methods primarily focus on the spatial domain, neglecting the understanding of frequency variations and ignoring the impact of implicit noise in skip connections. In this paper, we introduce a multi-scale frequency selection network (MSFSNet) that seamlessly integrates spatial and frequency domain knowledge, selectively recovering richer and more accurate information. Specifically, we initially capture spatial features and input them into dynamic filter selection modules (DFS) at different scales to integrate frequency knowledge. DFS utilizes learnable filters to generate high and low-frequency information and employs a frequency cross-attention mechanism (FCAM) to determine the most information to recover. To learn a multi-scale and accurate set of hybrid features, we develop a skip feature fusion block (SFF) that leverages contextual features to discriminatively determine which information should be propagated in skip-connections. It is worth noting that our DFS and SFF are generic plug-in modules that can be directly employed in existing networks without any adjustments, leading to performance improvements. Extensive experiments across various image restoration tasks demonstrate that our MSFSNet achieves performance that is either superior or comparable to state-of-the-art algorithms.