Uncertainty-Aware Source-Free Adaptive Image Super-Resolution with Wavelet Augmentation Transformer

TL;DR

This paper introduces SODA-SR, a source-free unsupervised domain adaptation framework for image super-resolution that uses wavelet-based augmentation and uncertainty-aware self-training to improve performance without source data.

Contribution

The paper proposes a novel source-free domain adaptation method for image super-resolution, incorporating wavelet augmentation and uncertainty estimation to enhance pseudo-label accuracy.

Findings

SODA-SR outperforms existing UDA methods in synthetic to real SR tasks.

The wavelet augmentation transformer effectively utilizes low-frequency information across samples.

Uncertainty-aware self-training improves pseudo-label reliability and SR quality.

Abstract

Unsupervised Domain Adaptation (UDA) can effectively address domain gap issues in real-world image Super-Resolution (SR) by accessing both the source and target data. Considering privacy policies or transmission restrictions of source data in practical scenarios, we propose a SOurce-free Domain Adaptation framework for image SR (SODA-SR) to address this issue, i.e., adapt a source-trained model to a target domain with only unlabeled target data. SODA-SR leverages the source-trained model to generate refined pseudo-labels for teacher-student learning. To better utilize pseudo-labels, we propose a novel wavelet-based augmentation method, named Wavelet Augmentation Transformer (WAT), which can be flexibly incorporated with existing networks, to implicitly produce useful augmented data. WAT learns low-frequency information of varying levels across diverse samples, which is aggregated efficiently via deformable attention. Furthermore, an uncertainty-aware self-training mechanism is proposed to improve the accuracy of pseudo-labels, with inaccurate predictions being rectified by uncertainty estimation. To acquire better SR results and avoid overfitting pseudo-labels, several regularization losses are proposed to constrain target LR and SR images in the frequency domain. Experiments show that without accessing source data, SODA-SR outperforms state-of-the-art UDA methods in both synthetic$\rightarrow$real and real$\rightarrow$real adaptation settings, and is not constrained by specific network architectures.