Exploring Dropout Discriminator for Domain Adaptation

TL;DR

This paper introduces a novel domain adaptation method using a dropout-based ensemble discriminator that improves the alignment of source and target distributions by leveraging distributional estimates rather than point estimates, leading to better performance.

Contribution

The paper proposes a curriculum-based dropout ensemble discriminator for adversarial domain adaptation, enhancing distributional alignment and gradient estimation over traditional single discriminators.

Findings

Outperforms state-of-the-art domain adaptation methods.

Ensemble discriminator improves feature alignment.

Gradually increasing variance benefits training stability.

Abstract

Adaptation of a classifier to new domains is one of the challenging problems in machine learning. This has been addressed using many deep and non-deep learning based methods. Among the methodologies used, that of adversarial learning is widely applied to solve many deep learning problems along with domain adaptation. These methods are based on a discriminator that ensures source and target distributions are close. However, here we suggest that rather than using a point estimate obtaining by a single discriminator, it would be useful if a distribution based on ensembles of discriminators could be used to bridge this gap. This could be achieved using multiple classifiers or using traditional ensemble methods. In contrast, we suggest that a Monte Carlo dropout based ensemble discriminator could suffice to obtain the distribution based discriminator. Specifically, we propose a curriculum based dropout discriminator that gradually increases the variance of the sample based distribution and the corresponding reverse gradients are used to align the source and target feature representations. An ensemble of discriminators helps the model to learn the data distribution efficiently. It also provides a better gradient estimates to train the feature extractor. The detailed results and thorough ablation analysis show that our model outperforms state-of-the-art results.