Continuous Expert Assembly: Instance-Conditioned Low-Rank Residuals for All-in-One Image Restoration

TL;DR

The paper introduces Continuous Expert Assembly (CEA), a dynamic, token-wise image restoration framework that adaptively combines expert components based on spatial features, improving performance on diverse degradations.

Contribution

CEA employs a novel Cross-Attention Hyper-Adapter for instance-conditioned low-rank routing, enabling flexible, efficient, and interpretable all-in-one image restoration without external prompts or static experts.

Findings

CEA outperforms existing methods on multiple benchmarks.

It achieves clearer gains on spatially varying degradations.

CEA maintains efficiency in parameters, FLOPs, and runtime.

Abstract

Real-world image degradation is often unknown, spatially non-uniform, and compositional, requiring all-in-one restoration models to adapt a single set of weights to diverse local corruption patterns without test-time degradation labels. Existing methods typically modulate a shared backbone with global prompts or degradation descriptors, or route features through predefined expert pools. However, compact global conditioning can bottleneck localized degradation evidence, while static expert routing may produce homogeneous updates or rely on unstable sparse assignments. We propose \textbf{Continuous Expert Assembly} (CEA), a token-wise dynamic parameterization framework for all-in-one image restoration. CEA employs a lightweight \textbf{Cross-Attention Hyper-Adapter} to probe intermediate spatial features and synthesize instance-conditioned low-rank routing bases and residual directions. Each spatial token then assembles its own residual update via dense signed dot-product affinities over the generated rank-wise components, avoiding external prompts, static expert banks, and discrete Top- selection. The resulting assembly rule also admits a linear-attention perspective, making its dense token-wise routing behavior transparent. Experiments on AIO-3, AIO-5, and CDD-11 show that CEA improves average restoration quality over strong prompt-, descriptor-, and expert-based baselines, with the clearest gains on spatially varying and compositional degradations, while maintaining favorable parameter, FLOP, and runtime efficiency.