Dataset and Neural Recurrent Sequence Labeling Model for Open-Domain Factoid Question Answering

TL;DR

This paper introduces a large-scale real-world QA dataset and a novel sequence labeling neural model that improves answer extraction efficiency and handles unseen answers effectively in open-domain factoid question answering.

Contribution

The paper presents a new dataset WebQA and a sequence labeling approach that outperforms existing methods in neural question answering tasks.

Findings

WebQA contains over 42k questions and 556k evidences.

The proposed model achieves an F1 score of 74.69% on WebQA.

Performance drops only 3.72 F1 points with character-based input.

Abstract

While question answering (QA) with neural network, i.e. neural QA, has achieved promising results in recent years, lacking of large scale real-word QA dataset is still a challenge for developing and evaluating neural QA system. To alleviate this problem, we propose a large scale human annotated real-world QA dataset WebQA with more than 42k questions and 556k evidences. As existing neural QA methods resolve QA either as sequence generation or classification/ranking problem, they face challenges of expensive softmax computation, unseen answers handling or separate candidate answer generation component. In this work, we cast neural QA as a sequence labeling problem and propose an end-to-end sequence labeling model, which overcomes all the above challenges. Experimental results on WebQA show that our model outperforms the baselines significantly with an F1 score of 74.69% with word-based input, and the performance drops only 3.72 F1 points with more challenging character-based input.