Structural Similarity-Inspired Unfolding for Lightweight Image Super-Resolution

TL;DR

This paper introduces a lightweight image super-resolution method called SSIU that combines unfolding optimization constrained by structural similarity with modules designed for efficiency, outperforming current models in accuracy and resource usage.

Contribution

The novel SSIU method integrates structural similarity constraints into an unfolding framework with specialized modules, achieving high performance with fewer parameters.

Findings

Outperforms state-of-the-art models in super-resolution accuracy.

Uses fewer parameters and less memory than existing methods.

Demonstrates effective integration of structural similarity in unfolding models.

Abstract

Major efforts in data-driven image super-resolution (SR) primarily focus on expanding the receptive field of the model to better capture contextual information. However, these methods are typically implemented by stacking deeper networks or leveraging transformer-based attention mechanisms, which consequently increases model complexity. In contrast, model-driven methods based on the unfolding paradigm show promise in improving performance while effectively maintaining model compactness through sophisticated module design. Based on these insights, we propose a Structural Similarity-Inspired Unfolding (SSIU) method for efficient image SR. This method is designed through unfolding an SR optimization function constrained by structural similarity, aiming to combine the strengths of both data-driven and model-driven approaches. Our model operates progressively following the unfolding paradigm. Each iteration consists of multiple Mixed-Scale Gating Modules (MSGM) and an Efficient Sparse Attention Module (ESAM). The former implements comprehensive constraints on features, including a structural similarity constraint, while the latter aims to achieve sparse activation. In addition, we design a Mixture-of-Experts-based Feature Selector (MoE-FS) that fully utilizes multi-level feature information by combining features from different steps. Extensive experiments validate the efficacy and efficiency of our unfolding-inspired network. Our model outperforms current state-of-the-art models, boasting lower parameter counts and reduced memory consumption. Our code will be available at: https://github.com/eezkni/SSIU