PromptCIR: Blind Compressed Image Restoration with Prompt Learning

TL;DR

PromptCIR introduces a prompt-learning-based approach for blind compressed image restoration that adaptively encodes compression information, enabling effective restoration across various compression levels with minimal parameter overhead.

Contribution

The paper proposes a novel prompt-learning-based network, PromptCIR, which uses dynamic prompts to encode compression information and adapt to different compression levels in blind image restoration.

Findings

Achieved first place in NTIRE 2024 blind compressed image enhancement challenge.

Validated effectiveness through extensive experiments.

Efficiently handles various compression levels with minimal additional parameters.

Abstract

Blind Compressed Image Restoration (CIR) has garnered significant attention due to its practical applications. It aims to mitigate compression artifacts caused by unknown quality factors, particularly with JPEG codecs. Existing works on blind CIR often seek assistance from a quality factor prediction network to facilitate their network to restore compressed images. However, the predicted numerical quality factor lacks spatial information, preventing network adaptability toward image contents. Recent studies in prompt-learning-based image restoration have showcased the potential of prompts to generalize across varied degradation types and degrees. This motivated us to design a prompt-learning-based compressed image restoration network, dubbed PromptCIR, which can effectively restore images from various compress levels. Specifically, PromptCIR exploits prompts to encode compression information implicitly, where prompts directly interact with soft weights generated from image features, thus providing dynamic content-aware and distortion-aware guidance for the restoration process. The light-weight prompts enable our method to adapt to different compression levels, while introducing minimal parameter overhead. Overall, PromptCIR leverages the powerful transformer-based backbone with the dynamic prompt module to proficiently handle blind CIR tasks, winning first place in the NTIRE 2024 challenge of blind compressed image enhancement track. Extensive experiments have validated the effectiveness of our proposed PromptCIR. The code is available at https://github.com/lbc12345/PromptCIR-NTIRE24.