Look-Around Before You Leap: High-Frequency Injected Transformer for Image Restoration

TL;DR

This paper introduces HIT, a high-frequency injected transformer that enhances local detail preservation in image restoration by combining high-frequency information injection, multi-scale feature aggregation, and spatial enhancement, outperforming state-of-the-art methods.

Contribution

The paper proposes HIT, a novel transformer architecture that effectively incorporates high-frequency details and multi-scale features for improved image restoration performance.

Findings

HIT achieves superior results across 9 diverse image restoration tasks.

The model maintains linear computational complexity.

Extensive experiments validate its effectiveness over existing methods.

Abstract

Transformer-based approaches have achieved superior performance in image restoration, since they can model long-term dependencies well. However, the limitation in capturing local information restricts their capacity to remove degradations. While existing approaches attempt to mitigate this issue by incorporating convolutional operations, the core component in Transformer, i.e., self-attention, which serves as a low-pass filter, could unintentionally dilute or even eliminate the acquired local patterns. In this paper, we propose HIT, a simple yet effective High-frequency Injected Transformer for image restoration. Specifically, we design a window-wise injection module (WIM), which incorporates abundant high-frequency details into the feature map, to provide reliable references for restoring high-quality images. We also develop a bidirectional interaction module (BIM) to aggregate features at different scales using a mutually reinforced paradigm, resulting in spatially and contextually improved representations. In addition, we introduce a spatial enhancement unit (SEU) to preserve essential spatial relationships that may be lost due to the computations carried out across channel dimensions in the BIM. Extensive experiments on 9 tasks (real noise, real rain streak, raindrop, motion blur, moir\'e, shadow, snow, haze, and low-light condition) demonstrate that HIT with linear computational complexity performs favorably against the state-of-the-art methods. The source code and pre-trained models will be available at https://github.com/joshyZhou/HIT.