A Balanced Neuro-Symbolic Approach for Commonsense Abductive Logic

TL;DR

This paper introduces a neuro-symbolic method that iteratively augments logical reasoning with commonsense knowledge from LLMs, improving formal reasoning performance on datasets with missing facts.

Contribution

It presents a novel feedback-driven approach that combines LLMs and logic solvers to handle incomplete information in reasoning tasks.

Findings

Significant performance improvements over existing methods.

Effective integration of neural and symbolic reasoning.

Robustness in datasets with missing commonsense facts.

Abstract

Although Large Language Models (LLMs) have demonstrated impressive formal reasoning abilities, they often break down when problems require complex proof planning. One promising approach for improving LLM reasoning abilities involves translating problems into formal logic and using a logic solver. Although off-the-shelf logic solvers are in principle substantially more efficient than LLMs at logical reasoning, they assume that all relevant facts are provided in a question and are unable to deal with missing commonsense relations. In this work, we propose a novel method that uses feedback from the logic solver to augment a logic problem with commonsense relations provided by the LLM, in an iterative manner. This involves a search procedure through potential commonsense assumptions to maximize the chance of finding useful facts while keeping cost tractable. On a collection of pure-logical reasoning datasets, from which some commonsense information has been removed, our method consistently achieves considerable improvements over existing techniques, demonstrating the value in balancing neural and symbolic elements when working in human contexts.