A BERT Baseline for the Natural Questions

TL;DR

This paper presents a BERT-based baseline for the Natural Questions dataset, significantly improving answer accuracy and providing resources for future research in question answering tasks.

Contribution

It introduces a BERT-based model that narrows the gap between previous results and human performance on Natural Questions.

Findings

30-50% relative improvement in F1 scores

Achieved state-of-the-art baseline results

Resources available for further research

Abstract

This technical note describes a new baseline for the Natural Questions. Our model is based on BERT and reduces the gap between the model F1 scores reported in the original dataset paper and the human upper bound by 30% and 50% relative for the long and short answer tasks respectively. This baseline has been submitted to the official NQ leaderboard at ai.google.com/research/NaturalQuestions. Code, preprocessed data and pretrained model are available at https://github.com/google-research/language/tree/master/language/question_answering/bert_joint.