Multi-Expert Adaptive Selection: Task-Balancing for All-in-One Image Restoration

TL;DR

This paper introduces a multi-expert adaptive selection framework for multi-task image restoration, dynamically selecting experts based on image content to improve performance and resource efficiency.

Contribution

It proposes a novel multi-expert selection and ensemble scheme that adaptively balances multiple image restoration tasks by leveraging content-aware expert selection.

Findings

Outperforms existing methods in multi-task image restoration

Effectively balances and optimizes multiple tasks simultaneously

Reduces interference from irrelevant experts

Abstract

The use of a single image restoration framework to achieve multi-task image restoration has garnered significant attention from researchers. However, several practical challenges remain, including meeting the specific and simultaneous demands of different tasks, balancing relationships between tasks, and effectively utilizing task correlations in model design. To address these challenges, this paper explores a multi-expert adaptive selection mechanism. We begin by designing a feature representation method that accounts for both the pixel channel level and the global level, encompassing low-frequency and high-frequency components of the image. Based on this method, we construct a multi-expert selection and ensemble scheme. This scheme adaptively selects the most suitable expert from the expert library according to the content of the input image and the prompts of the current task. It not only meets the individualized needs of different tasks but also achieves balance and optimization across tasks. By sharing experts, our design promotes interconnections between different tasks, thereby enhancing overall performance and resource utilization. Additionally, the multi-expert mechanism effectively eliminates irrelevant experts, reducing interference from them and further improving the effectiveness and accuracy of image restoration. Experimental results demonstrate that our proposed method is both effective and superior to existing approaches, highlighting its potential for practical applications in multi-task image restoration.