Learning from a Generative Oracle: Domain Adaptation for Restoration

TL;DR

LEGO is a three-stage framework that enables pre-trained image restoration models to adapt to new, real-world domains without paired data, by generating pseudo-ground-truths from a generative oracle and fine-tuning the model.

Contribution

We introduce LEGO, a novel post-training domain adaptation method that transforms unsupervised data into pseudo-supervised pairs using a generative oracle, avoiding architectural changes.

Findings

Significant performance improvements on real-world benchmarks.

Effective domain adaptation without paired data or model architecture modifications.

Maintains original model robustness after adaptation.

Abstract

Pre-trained image restoration models often fail on real-world, out-of-distribution degradations due to significant domain gaps. Adapting to these unseen domains is challenging, as out-of-distribution data lacks ground truth, and traditional adaptation methods often require complex architectural changes. We propose LEGO (Learning from a Generative Oracle), a practical three-stage framework for post-training domain adaptation without paired data. LEGO converts this unsupervised challenge into a tractable pseudo-supervised one. First, we obtain initial restorations from the pre-trained model. Second, we leverage a frozen, large-scale generative oracle to refine these estimates into high-quality pseudo-ground-truths. Third, we fine-tune the original model using a mixed-supervision strategy combining in-distribution data with these new pseudo-pairs. This approach adapts the model to the new distribution without sacrificing its original robustness or requiring architectural modifications. Experiments demonstrate that LEGO effectively bridges the domain gap, significantly improving performance on diverse real-world benchmarks.