Natural Language Deduction with Incomplete Information

TL;DR

This paper introduces a system for natural language deduction that can handle incomplete information by abductively inferring missing premises, enabling more flexible reasoning in question-answering and claim verification tasks.

Contribution

It proposes a bidirectional search method combining deductive and abductive reasoning with validation, allowing inference with incomplete premises in natural language.

Findings

Successfully infers missing premises in in- and out-of-domain datasets

Outperforms baseline methods on modified EntailmentBank and Everyday Norms datasets

Demonstrates the effectiveness of abductive reasoning with validation in natural language inference

Abstract

A growing body of work studies how to answer a question or verify a claim by generating a natural language "proof": a chain of deductive inferences yielding the answer based on a set of premises. However, these methods can only make sound deductions when they follow from evidence that is given. We propose a new system that can handle the underspecified setting where not all premises are stated at the outset; that is, additional assumptions need to be materialized to prove a claim. By using a natural language generation model to abductively infer a premise given another premise and a conclusion, we can impute missing pieces of evidence needed for the conclusion to be true. Our system searches over two fringes in a bidirectional fashion, interleaving deductive (forward-chaining) and abductive (backward-chaining) generation steps. We sample multiple possible outputs for each step to achieve coverage of the search space, at the same time ensuring correctness by filtering low-quality generations with a round-trip validation procedure. Results on a modified version of the EntailmentBank dataset and a new dataset called Everyday Norms: Why Not? show that abductive generation with validation can recover premises across in- and out-of-domain settings.