Adversarial Domain Adaptation for Machine Reading Comprehension

TL;DR

This paper introduces AdaMRC, an adversarial domain adaptation framework for machine reading comprehension that generates pseudo questions and learns domain-invariant representations, improving cross-domain performance without labeled target data.

Contribution

The paper proposes a novel adversarial domain adaptation method for MRC that generates pseudo questions and enforces domain-invariant features, applicable to various models and pre-trained language models.

Findings

Effective domain adaptation across different datasets

Compatibility with large-scale pre-trained models like BERT and ELMo

Extension to semi-supervised learning scenarios

Abstract

In this paper, we focus on unsupervised domain adaptation for Machine Reading Comprehension (MRC), where the source domain has a large amount of labeled data, while only unlabeled passages are available in the target domain. To this end, we propose an Adversarial Domain Adaptation framework (AdaMRC), where ($i$) pseudo questions are first generated for unlabeled passages in the target domain, and then ($ii$) a domain classifier is incorporated into an MRC model to predict which domain a given passage-question pair comes from. The classifier and the passage-question encoder are jointly trained using adversarial learning to enforce domain-invariant representation learning. Comprehensive evaluations demonstrate that our approach ($i$) is generalizable to different MRC models and datasets, ($ii$) can be combined with pre-trained large-scale language models (such as ELMo and BERT), and ($iii$) can be extended to semi-supervised learning.