Data Augmentation for BERT Fine-Tuning in Open-Domain Question Answering

TL;DR

This paper introduces a data augmentation method using distant supervision to improve BERT fine-tuning for open-domain question answering, achieving significant performance gains on English and Chinese datasets.

Contribution

It proposes a stage-wise fine-tuning approach with data augmentation that leverages positive and negative examples, setting new benchmarks in QA performance.

Findings

Large improvements over previous methods on English QA datasets

Established new baselines on Chinese QA datasets

Effective use of distant supervision for data augmentation

Abstract

Recently, a simple combination of passage retrieval using off-the-shelf IR techniques and a BERT reader was found to be very effective for question answering directly on Wikipedia, yielding a large improvement over the previous state of the art on a standard benchmark dataset. In this paper, we present a data augmentation technique using distant supervision that exploits positive as well as negative examples. We apply a stage-wise approach to fine tuning BERT on multiple datasets, starting with data that is "furthest" from the test data and ending with the "closest". Experimental results show large gains in effectiveness over previous approaches on English QA datasets, and we establish new baselines on two recent Chinese QA datasets.