Modular Domain Adaptation

TL;DR

This paper proposes a modular approach to domain adaptation in text classification, enabling independent model producers and consumers to improve out-of-domain accuracy through lightweight techniques.

Contribution

It introduces a modular framework for domain adaptation with two new lightweight techniques, enhancing out-of-domain performance without access to source data.

Findings

Reliable increase in out-of-domain accuracy across four datasets

Effective with both linear and contextual embedding models

Provides practical recommendations and open-source tools

Abstract

Off-the-shelf models are widely used by computational social science researchers to measure properties of text, such as sentiment. However, without access to source data it is difficult to account for domain shift, which represents a threat to validity. Here, we treat domain adaptation as a modular process that involves separate model producers and model consumers, and show how they can independently cooperate to facilitate more accurate measurements of text. We introduce two lightweight techniques for this scenario, and demonstrate that they reliably increase out-of-domain accuracy on four multi-domain text classification datasets when used with linear and contextual embedding models. We conclude with recommendations for model producers and consumers, and release models and replication code to accompany this paper.