Learning to Generalize for Cross-domain QA

TL;DR

This paper introduces a cost-effective method combining prompting and linear probing to improve cross-domain question-answering models' generalization, outperforming existing baselines without additional training costs.

Contribution

It proposes a novel, cost-efficient approach that enhances out-of-domain QA performance by integrating prompting with linear probing and fine-tuning strategies.

Findings

Outperforms state-of-the-art baselines with 4.5%-7.9% F1 score increase

Effective for both generative and discriminative models

Easily integrated into pre-trained models

Abstract

There have been growing concerns regarding the out-of-domain generalization ability of natural language processing (NLP) models, particularly in question-answering (QA) tasks. Current synthesized data augmentation methods for QA are hampered by increased training costs. To address this issue, we propose a novel approach that combines prompting methods and linear probing then fine-tuning strategy, which does not entail additional cost. Our method has been theoretically and empirically shown to be effective in enhancing the generalization ability of both generative and discriminative models. Our approach outperforms state-of-the-art baselines, with an average increase in F1 score of 4.5%-7.9%. Furthermore, our method can be easily integrated into any pre-trained models and offers a promising solution to the under-explored cross-domain QA task. We release our source code at GitHub*.