Leveraging LLMs for Hypothetical Deduction in Logical Inference: A Neuro-Symbolic Approach

TL;DR

This paper introduces LINA, a neuro-symbolic approach leveraging LLMs for logical reasoning that improves robustness and independence from external solvers, outperforming existing methods across multiple tasks.

Contribution

LINA enables autonomous propositional logic reasoning within LLMs, eliminating external dependencies and effectively handling diverse question features.

Findings

LINA outperforms propositional logic frameworks and prompting methods.

Achieves 24.34% improvement over LINC on FOLIO.

Surpasses CoT and CoT-SC prompting strategies by up to 24.02%.

Abstract

Large Language Models (LLMs) have exhibited remarkable potential across a wide array of reasoning tasks, including logical reasoning. Although massive efforts have been made to empower the logical reasoning ability of LLMs via external logical symbolic solvers, crucial challenges of the poor generalization ability to questions with different features and inevitable question information loss of symbolic solver-driven approaches remain unresolved. To mitigate these issues, we introduce LINA, a LLM-driven neuro-symbolic approach for faithful logical reasoning. By enabling an LLM to autonomously perform the transition from propositional logic extraction to sophisticated logical reasoning, LINA not only bolsters the resilience of the reasoning process but also eliminates the dependency on external solvers. Additionally, through its adoption of a hypothetical-deductive reasoning paradigm, LINA effectively circumvents the expansive search space challenge that plagues traditional forward reasoning methods. Empirical evaluations demonstrate that LINA substantially outperforms both established propositional logic frameworks and conventional prompting techniques across a spectrum of five logical reasoning tasks. Specifically, LINA achieves an improvement of 24.34% over LINC on the FOLIO dataset, while also surpassing prompting strategies like CoT and CoT-SC by up to 24.02%. Our code is available at https://github.com/wufeiwuwoshihua/nshy.