Domain Adaptation for Statistical Classifiers

TL;DR

This paper addresses domain adaptation in statistical classifiers, proposing a mixture model approach with efficient inference algorithms, demonstrating improved NLP task performance when in-domain labeled data is scarce.

Contribution

Introduces a mixture model framework for domain adaptation in classifiers and develops efficient inference algorithms for practical implementation.

Findings

Improved classifier performance on NLP tasks.

Effective adaptation with limited in-domain data.

Validated on multiple real-world datasets.

Abstract

The most basic assumption used in statistical learning theory is that training data and test data are drawn from the same underlying distribution. Unfortunately, in many applications, the "in-domain" test data is drawn from a distribution that is related, but not identical, to the "out-of-domain" distribution of the training data. We consider the common case in which labeled out-of-domain data is plentiful, but labeled in-domain data is scarce. We introduce a statistical formulation of this problem in terms of a simple mixture model and present an instantiation of this framework to maximum entropy classifiers and their linear chain counterparts. We present efficient inference algorithms for this special case based on the technique of conditional expectation maximization. Our experimental results show that our approach leads to improved performance on three real world tasks on four different data sets from the natural language processing domain.