Is user feedback always informative? Retrieval Latent Defending for Semi-Supervised Domain Adaptation without Source Data

TL;DR

This paper introduces Retrieval Latent Defending, a novel method to improve semi-supervised domain adaptation by addressing negatively biased user feedback, leading to better performance across multiple tasks.

Contribution

It identifies the issue of negatively biased feedback in semi-supervised domain adaptation and proposes a scalable approach to mitigate this bias using latent defending samples.

Findings

Significant performance improvements on image classification, semantic segmentation, and medical imaging tasks.

Effectiveness of the approach demonstrated across various state-of-the-art SemiSDA methods.

Analysis of negatively biased feedback and its impact on adaptation performance.

Abstract

This paper aims to adapt the source model to the target environment, leveraging small user feedback (i.e., labeled target data) readily available in real-world applications. We find that existing semi-supervised domain adaptation (SemiSDA) methods often suffer from poorly improved adaptation performance when directly utilizing such feedback data, as shown in Figure 1. We analyze this phenomenon via a novel concept called Negatively Biased Feedback (NBF), which stems from the observation that user feedback is more likely for data points where the model produces incorrect predictions. To leverage this feedback while avoiding the issue, we propose a scalable adapting approach, Retrieval Latent Defending. This approach helps existing SemiSDA methods to adapt the model with a balanced supervised signal by utilizing latent defending samples throughout the adaptation process. We demonstrate the problem caused by NBF and the efficacy of our approach across various benchmarks, including image classification, semantic segmentation, and a real-world medical imaging application. Our extensive experiments reveal that integrating our approach with multiple state-of-the-art SemiSDA methods leads to significant performance improvements.