Omni Aggregation Networks for Lightweight Image Super-Resolution

TL;DR

This paper introduces Omni-SR, a lightweight image super-resolution architecture with omni-directional self-attention and multi-scale interaction, significantly improving performance while maintaining efficiency.

Contribution

The work proposes the Omni Self-Attention block and omni-scale aggregation scheme, enabling comprehensive spatial and channel interactions in lightweight super-resolution models.

Findings

Achieves record-high performance on lightweight SR benchmarks

Omni-SR outperforms existing models with fewer parameters

Demonstrates effective modeling of spatial and channel correlations

Abstract

While lightweight ViT framework has made tremendous progress in image super-resolution, its uni-dimensional self-attention modeling, as well as homogeneous aggregation scheme, limit its effective receptive field (ERF) to include more comprehensive interactions from both spatial and channel dimensions. To tackle these drawbacks, this work proposes two enhanced components under a new Omni-SR architecture. First, an Omni Self-Attention (OSA) block is proposed based on dense interaction principle, which can simultaneously model pixel-interaction from both spatial and channel dimensions, mining the potential correlations across omni-axis (i.e., spatial and channel). Coupling with mainstream window partitioning strategies, OSA can achieve superior performance with compelling computational budgets. Second, a multi-scale interaction scheme is proposed to mitigate sub-optimal ERF (i.e., premature saturation) in shallow models, which facilitates local propagation and meso-/global-scale interactions, rendering an omni-scale aggregation building block. Extensive experiments demonstrate that Omni-SR achieves record-high performance on lightweight super-resolution benchmarks (e.g., 26.95 dB@Urban100 $\times 4$ with only 792K parameters). Our code is available at \url{https://github.com/Francis0625/Omni-SR}.