Sound and Complete Neurosymbolic Reasoning with LLM-Grounded Interpretations

TL;DR

This paper introduces a neurosymbolic reasoning method that integrates large language models into formal logic interpretation, ensuring logical soundness and completeness while leveraging LLMs' knowledge.

Contribution

It provides a theoretical framework for neurosymbolic reasoning that combines LLMs with formal logic, maintaining logical properties unlike prior approaches.

Findings

Experimental validation on factuality benchmarks shows promising results.

The method preserves soundness and completeness in reasoning.

Demonstrates feasibility of LLM-grounded formal semantics.

Abstract

Large language models (LLMs) have demonstrated impressive capabilities in natural language understanding and generation, but they exhibit problems with logical consistency in the output they generate. How can we harness LLMs' broad-coverage parametric knowledge in formal reasoning despite their inconsistency? We present a method for directly integrating an LLM into the interpretation function of the formal semantics for a paraconsistent logic. We provide experimental evidence for the feasibility of the method by evaluating the function using datasets created from several short-form factuality benchmarks. Unlike prior work, our method offers a theoretical framework for neurosymbolic reasoning that leverages an LLM's knowledge while preserving the underlying logic's soundness and completeness properties.