Abductive Reasoning with Probabilistic Commonsense

TL;DR

This paper introduces PACS, a probabilistic framework that models individual differences in commonsense beliefs to improve abductive reasoning in LLMs, outperforming previous methods.

Contribution

It proposes a novel probabilistic abductive reasoning algorithm that explicitly accounts for variability in commonsense beliefs among individuals.

Findings

PACS outperforms chain-of-thought reasoning on multiple benchmarks.

PACS surpasses prior neurosymbolic and search-based methods.

The framework effectively models individual differences in commonsense beliefs.

Abstract

Recent efforts to improve the reasoning abilities of Large Language Models (LLMs) have focused on integrating formal logic solvers within neurosymbolic frameworks. A key challenge is that formal solvers lack commonsense world knowledge, preventing them from making reasoning steps that humans find obvious. Prior methods address this by using LLMs to supply missing commonsense assumptions, but these approaches implicitly assume universal agreement on such commonsense facts. In reality, commonsense beliefs vary across individuals. We propose a probabilistic framework for abductive commonsense reasoning that explicitly models this variation, aiming to determine whether most people would judge a statement as true or false. We introduce Probabilistic Abductive CommonSense (PACS), a novel algorithm that uses an LLM and a formal solver to sample proofs as observations of individuals' distinct commonsense beliefs, and aggregates conclusions across these samples. Empirically, PACS outperforms chain-of-thought reasoning, prior neurosymbolic methods, and search-based approaches across multiple benchmarks.