Towards Graph-hop Retrieval and Reasoning in Complex Question Answering over Textual Database

TL;DR

This paper introduces Graph-Hop, a new paradigm for multi-chain, multi-hop reasoning in complex textual question answering, supported by a novel benchmark and a graph retrieval method, advancing interpretability and reasoning diversity.

Contribution

It proposes Graph-Hop for multi-hop reasoning, constructs the ReasonGraphQA benchmark with detailed evidence graphs, and introduces the Bidirectional Graph Retrieval method for evidence explanation.

Findings

Graph-Hop improves reasoning diversity and scale.

ReasonGraphQA provides fine-grained evidence graphs.

Existing models have limitations in complex reasoning.

Abstract

In Textual question answering (TQA) systems, complex questions often require retrieving multiple textual fact chains with multiple reasoning steps. While existing benchmarks are limited to single-chain or single-hop retrieval scenarios. In this paper, we propose to conduct Graph-Hop -- a novel multi-chains and multi-hops retrieval and reasoning paradigm in complex question answering. We construct a new benchmark called ReasonGraphQA, which provides explicit and fine-grained evidence graphs for complex questions to support interpretable reasoning, comprehensive and detailed reasoning. And ReasonGraphQA also shows an advantage in reasoning diversity and scale. Moreover, We propose a strong graph-hop baseline called Bidirectional Graph Retrieval (BGR) method for generating an explanation graph of textual evidence in knowledge reasoning and question answering. We have thoroughly evaluated existing evidence retrieval and reasoning models on the ReasonGraphQA. Experiments highlight Graph-Hop is a promising direction for answering complex questions, but it still has certain limitations. We have further studied mitigation strategies to meet these challenges and discuss future directions.