Entailment Tree Explanations via Iterative Retrieval-Generation Reasoner

TL;DR

This paper introduces IRGR, a novel model that iteratively retrieves and generates structured entailment trees to improve explainability in question answering systems, significantly outperforming previous methods.

Contribution

The paper presents IRGR, a new architecture combining retrieval and generation for entailment trees, enhancing explainability and performance in QA explanations.

Findings

Outperforms existing benchmarks on EntailmentBank dataset

Achieves around 300% gain in overall correctness

Effectively leverages intermediate conclusions in explanations

Abstract

Large language models have achieved high performance on various question answering (QA) benchmarks, but the explainability of their output remains elusive. Structured explanations, called entailment trees, were recently suggested as a way to explain and inspect a QA system's answer. In order to better generate such entailment trees, we propose an architecture called Iterative Retrieval-Generation Reasoner (IRGR). Our model is able to explain a given hypothesis by systematically generating a step-by-step explanation from textual premises. The IRGR model iteratively searches for suitable premises, constructing a single entailment step at a time. Contrary to previous approaches, our method combines generation steps and retrieval of premises, allowing the model to leverage intermediate conclusions, and mitigating the input size limit of baseline encoder-decoder models. We conduct experiments using the EntailmentBank dataset, where we outperform existing benchmarks on premise retrieval and entailment tree generation, with around 300% gain in overall correctness.