Cheap and Good? Simple and Effective Data Augmentation for Low Resource Machine Reading

TL;DR

This paper introduces a simple data augmentation method for low-resource machine reading comprehension that pretrains answer extraction components on augmented data with approximate answer contexts, significantly improving performance.

Contribution

The paper presents a novel, straightforward data augmentation strategy that enhances low-resource MRC by pretraining answer extractors on approximate answer contexts before fine-tuning.

Findings

Significant improvement in BERT retriever performance (15.12%) on TechQA.

Answer extractor F1 score increased by 4.33% on TechQA.

Up to 3.9% increase in exact match on PolicyQA.

Abstract

We propose a simple and effective strategy for data augmentation for low-resource machine reading comprehension (MRC). Our approach first pretrains the answer extraction components of a MRC system on the augmented data that contains approximate context of the correct answers, before training it on the exact answer spans. The approximate context helps the QA method components in narrowing the location of the answers. We demonstrate that our simple strategy substantially improves both document retrieval and answer extraction performance by providing larger context of the answers and additional training data. In particular, our method significantly improves the performance of BERT based retriever (15.12\%), and answer extractor (4.33\% F1) on TechQA, a complex, low-resource MRC task. Further, our data augmentation strategy yields significant improvements of up to 3.9\% exact match (EM) and 2.7\% F1 for answer extraction on PolicyQA, another practical but moderate sized QA dataset that also contains long answer spans.