Neural Unsupervised Domain Adaptation in NLP---A Survey

TL;DR

This survey reviews neural unsupervised domain adaptation techniques in NLP, highlighting methods, challenges, and future directions for models adapting to new, unlabeled domains without labeled target data.

Contribution

It provides a comprehensive overview of neural unsupervised domain adaptation methods in NLP, from traditional to pre-trained models, and discusses biases and future research directions.

Findings

Traditional methods and pre-trained transfer are key approaches.

Most attention has been on specific NLP tasks.

Future needs include out-of-distribution generalization.

Abstract

Deep neural networks excel at learning from labeled data and achieve state-of-the-art resultson a wide array of Natural Language Processing tasks. In contrast, learning from unlabeled data, especially under domain shift, remains a challenge. Motivated by the latest advances, in this survey we review neural unsupervised domain adaptation techniques which do not require labeled target domain data. This is a more challenging yet a more widely applicable setup. We outline methods, from early traditional non-neural methods to pre-trained model transfer. We also revisit the notion of domain, and we uncover a bias in the type of Natural Language Processing tasks which received most attention. Lastly, we outline future directions, particularly the broader need for out-of-distribution generalization of future NLP.