LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restoration

TL;DR

LoRA-IR introduces a dynamic, low-rank expert-based framework that enhances all-in-one image restoration by adaptively handling diverse real-world degradations with state-of-the-art performance and efficiency.

Contribution

The paper proposes LoRA-IR, a novel framework combining degradation-guided pre-training and low-rank fine-tuning with Mixture-of-Experts for versatile image restoration.

Findings

Achieves SOTA results across 14 IR tasks and 29 benchmarks.

Maintains high computational efficiency.

Effectively handles complex, unknown degradations in real-world scenarios.

Abstract

Prompt-based all-in-one image restoration (IR) frameworks have achieved remarkable performance by incorporating degradation-specific information into prompt modules. Nevertheless, handling the complex and diverse degradations encountered in real-world scenarios remains a significant challenge. To tackle this, we propose LoRA-IR, a flexible framework that dynamically leverages compact low-rank experts to facilitate efficient all-in-one image restoration. Specifically, LoRA-IR consists of two training stages: degradation-guided pre-training and parameter-efficient fine-tuning. In the pre-training stage, we enhance the pre-trained CLIP model by introducing a simple mechanism that scales it to higher resolutions, allowing us to extract robust degradation representations that adaptively guide the IR network. In the fine-tuning stage, we refine the pre-trained IR network through low-rank adaptation (LoRA). Built upon a Mixture-of-Experts (MoE) architecture, LoRA-IR dynamically integrates multiple low-rank restoration experts through a degradation-guided router. This dynamic integration mechanism significantly enhances our model's adaptability to diverse and unknown degradations in complex real-world scenarios. Extensive experiments demonstrate that LoRA-IR achieves SOTA performance across 14 IR tasks and 29 benchmarks, while maintaining computational efficiency. Code and pre-trained models will be available at: https://github.com/shallowdream204/LoRA-IR.