Efficient and Robust Question Answering from Minimal Context over Documents

TL;DR

This paper introduces a sentence selection approach for question answering that reduces computational costs and improves robustness by focusing on minimal relevant context, achieving comparable or better accuracy on multiple datasets.

Contribution

The paper proposes a simple sentence selector that identifies minimal context for QA, significantly reducing training and inference times while maintaining or improving accuracy and robustness.

Findings

Up to 15x reduction in training time

Up to 13x reduction in inference time

Enhanced robustness to adversarial inputs

Abstract

Neural models for question answering (QA) over documents have achieved significant performance improvements. Although effective, these models do not scale to large corpora due to their complex modeling of interactions between the document and the question. Moreover, recent work has shown that such models are sensitive to adversarial inputs. In this paper, we study the minimal context required to answer the question, and find that most questions in existing datasets can be answered with a small set of sentences. Inspired by this observation, we propose a simple sentence selector to select the minimal set of sentences to feed into the QA model. Our overall system achieves significant reductions in training (up to 15 times) and inference times (up to 13 times), with accuracy comparable to or better than the state-of-the-art on SQuAD, NewsQA, TriviaQA and SQuAD-Open. Furthermore, our experimental results and analyses show that our approach is more robust to adversarial inputs.