Atomic Inference for NLI with Generated Facts as Atoms

TL;DR

This paper explores atomic inference for natural language inference, using large language model-generated facts as atoms, and demonstrates that a multi-stage generation and training process improves interpretability and performance.

Contribution

It introduces a novel atomic inference approach utilizing LLM-generated facts as atoms, with a multi-stage process and training regime to enhance NLI performance.

Findings

Fact-based atomic inference outperforms other methods.

Multi-stage fact generation improves accuracy.

Training with facts enhances interpretability.

Abstract

With recent advances, neural models can achieve human-level performance on various natural language tasks. However, there are no guarantees that any explanations from these models are faithful, i.e. that they reflect the inner workings of the model. Atomic inference overcomes this issue, providing interpretable and faithful model decisions. This approach involves making predictions for different components (or atoms) of an instance, before using interpretable and deterministic rules to derive the overall prediction based on the individual atom-level predictions. We investigate the effectiveness of using LLM-generated facts as atoms, decomposing Natural Language Inference premises into lists of facts. While directly using generated facts in atomic inference systems can result in worse performance, with 1) a multi-stage fact generation process, and 2) a training regime that incorporates the facts, our fact-based method outperforms other approaches.