Open-domain Question Answering via Chain of Reasoning over Heterogeneous Knowledge

TL;DR

This paper introduces a novel open-domain question answering framework that employs intermediary reasoning modules to connect heterogeneous knowledge sources, significantly improving performance over previous models.

Contribution

It presents a new reasoning-based approach with intermediary modules that organize evidence into chains, enhancing retrieval and answer accuracy in ODQA tasks.

Findings

Achieves competitive results on OTT-QA and NQ datasets.

Substantially outperforms previous state-of-the-art on OTT-QA with 47.3% exact match.

Demonstrates effectiveness of reasoning over heterogeneous knowledge sources.

Abstract

We propose a novel open-domain question answering (ODQA) framework for answering single/multi-hop questions across heterogeneous knowledge sources. The key novelty of our method is the introduction of the intermediary modules into the current retriever-reader pipeline. Unlike previous methods that solely rely on the retriever for gathering all evidence in isolation, our intermediary performs a chain of reasoning over the retrieved set. Specifically, our method links the retrieved evidence with its related global context into graphs and organizes them into a candidate list of evidence chains. Built upon pretrained language models, our system achieves competitive performance on two ODQA datasets, OTT-QA and NQ, against tables and passages from Wikipedia. In particular, our model substantially outperforms the previous state-of-the-art on OTT-QA with an exact match score of 47.3 (45 % relative gain).