Leveraging Structured Information for Explainable Multi-hop Question Answering and Reasoning

TL;DR

This paper explores using structured semantic graphs extracted from text to improve multi-hop question answering, resulting in more faithful reasoning, better performance, and more interpretable explanations compared to traditional chain-of-thought methods.

Contribution

It introduces a framework that leverages extracted semantic structures for multi-hop QA, enhancing reasoning faithfulness and interpretability over existing methods.

Findings

Improved QA performance on benchmark datasets.

Generated reasoning chains are more faithful and interpretable.

Extracted structures provide grounded explanations preferred by humans.

Abstract

Neural models, including large language models (LLMs), achieve superior performance on multi-hop question-answering. To elicit reasoning capabilities from LLMs, recent works propose using the chain-of-thought (CoT) mechanism to generate both the reasoning chain and the answer, which enhances the model's capabilities in conducting multi-hop reasoning. However, several challenges still remain: such as struggling with inaccurate reasoning, hallucinations, and lack of interpretability. On the other hand, information extraction (IE) identifies entities, relations, and events grounded to the text. The extracted structured information can be easily interpreted by humans and machines (Grishman, 2019). In this work, we investigate constructing and leveraging extracted semantic structures (graphs) for multi-hop question answering, especially the reasoning process. Empirical results and human evaluations show that our framework: generates more faithful reasoning chains and substantially improves the QA performance on two benchmark datasets. Moreover, the extracted structures themselves naturally provide grounded explanations that are preferred by humans, as compared to the generated reasoning chains and saliency-based explanations.