Learning Domain-Aware Task Prompt Representations for Multi-Domain All-in-One Image Restoration

TL;DR

This paper introduces a novel multi-domain all-in-one image restoration method that adaptively combines task and domain prompts, significantly improving performance and generalization across various image restoration tasks and domains.

Contribution

The paper proposes the first multi-domain all-in-one image restoration framework using domain-aware task prompt representation learning, with adaptive prompt selection and domain priors distillation.

Findings

Outperforms state-of-the-art methods on multiple image restoration tasks.

Demonstrates strong generalization across diverse domains.

Effectively leverages multimodal large language models for domain priors.

Abstract

Recently, significant breakthroughs have been made in all-in-one image restoration (AiOIR), which can handle multiple restoration tasks with a single model. However, existing methods typically focus on a specific image domain, such as natural scene, medical imaging, or remote sensing. In this work, we aim to extend AiOIR to multiple domains and propose the first multi-domain all-in-one image restoration method, DATPRL-IR, based on our proposed Domain-Aware Task Prompt Representation Learning. Specifically, we first construct a task prompt pool containing multiple task prompts, in which task-related knowledge is implicitly encoded. For each input image, the model adaptively selects the most relevant task prompts and composes them into an instance-level task representation via a prompt composition mechanism (PCM). Furthermore, to endow the model with domain awareness, we introduce another domain prompt pool and distill domain priors from multimodal large language models into the domain prompts. PCM is utilized to combine the adaptively selected domain prompts into a domain representation for each input image. Finally, the two representations are fused to form a domain-aware task prompt representation which can make full use of both specific and shared knowledge across tasks and domains to guide the subsequent restoration process. Extensive experiments demonstrate that our DATPRL-IR significantly outperforms existing SOTA image restoration methods, while exhibiting strong generalization capabilities. Code is available at https://github.com/GuangluDong0728/DATPRL-IR.