Natural Language Deduction through Search over Statement Compositions

TL;DR

This paper introduces a search-based natural language deduction system that constructs reasoning trees to verify hypotheses from premises, improving interpretability and accuracy over end-to-end models.

Contribution

It presents a novel decompositional approach for natural language deduction that generates reasoning trees, enhancing interpretability and proof validity.

Findings

Successfully proves true statements in EntailmentBank dataset

Rejects false statements effectively

Produces explanations with 17% higher step validity

Abstract

In settings from fact-checking to question answering, we frequently want to know whether a collection of evidence (premises) entails a hypothesis. Existing methods primarily focus on the end-to-end discriminative version of this task, but less work has treated the generative version in which a model searches over the space of statements entailed by the premises to constructively derive the hypothesis. We propose a system for doing this kind of deductive reasoning in natural language by decomposing the task into separate steps coordinated by a search procedure, producing a tree of intermediate conclusions that faithfully reflects the system's reasoning process. Our experiments on the EntailmentBank dataset (Dalvi et al., 2021) demonstrate that the proposed system can successfully prove true statements while rejecting false ones. Moreover, it produces natural language explanations with a 17% absolute higher step validity than those produced by an end-to-end T5 model.