Evidence Aggregation for Answer Re-Ranking in Open-Domain Question Answering

TL;DR

This paper introduces two evidence aggregation models for answer re-ranking in open-domain QA, improving answer accuracy by combining information from multiple passages, and achieves state-of-the-art results on three datasets.

Contribution

It proposes strength-based and coverage-based re-ranking methods that leverage multiple passages for better answer selection in open-domain QA.

Findings

Achieved state-of-the-art results on Quasar-T, SearchQA, and TriviaQA datasets.

Improved answer accuracy by about 8 percentage points on two datasets.

Demonstrated the effectiveness of evidence aggregation in answer re-ranking.

Abstract

A popular recent approach to answering open-domain questions is to first search for question-related passages and then apply reading comprehension models to extract answers. Existing methods usually extract answers from single passages independently. But some questions require a combination of evidence from across different sources to answer correctly. In this paper, we propose two models which make use of multiple passages to generate their answers. Both use an answer-reranking approach which reorders the answer candidates generated by an existing state-of-the-art QA model. We propose two methods, namely, strength-based re-ranking and coverage-based re-ranking, to make use of the aggregated evidence from different passages to better determine the answer. Our models have achieved state-of-the-art results on three public open-domain QA datasets: Quasar-T, SearchQA and the open-domain version of TriviaQA, with about 8 percentage points of improvement over the former two datasets.