LLMs as ASP Programmers: Self-Correction Enables Task-Agnostic Nonmonotonic Reasoning

TL;DR

This paper introduces LLM+ASP, a framework that enables large language models to perform nonmonotonic reasoning using Answer Set Programming, with self-correction improving accuracy across diverse benchmarks.

Contribution

The framework operates without per-task engineering, leveraging self-correction and ASP to enhance reasoning capabilities of LLMs across various tasks.

Findings

Stable model semantics enable natural expression of defaults and exceptions.

Self-correction significantly improves performance, reducing reliance on handcrafted knowledge.

Concise in-context guides outperform verbose documentation, highlighting a 'context rot' effect.

Abstract

Recent large language models (LLMs) have achieved impressive reasoning milestones but continue to struggle with high computational costs, logical inconsistencies, and sharp performance degradation on high-complexity problems. While neuro-symbolic methods attempt to mitigate these issues by coupling LLMs with symbolic reasoners, existing approaches typically rely on monotonic logics (e.g., SMT) that cannot represent defeasible reasoning -- essential components of human cognition. We present "LLM+ASP," a framework that translates natural language into Answer Set Programming (ASP), a nonmonotonic formalism based on stable model semantics. Unlike prior "LLM+ASP" approaches that require manually authored knowledge modules, domain-specific prompts, or evaluation restricted to single problem classes, our framework operates without any per-task engineering and applies uniformly across diverse reasoning tasks. Our system utilizes an automated self-correction loop where structured feedback from the ASP solver enables iterative refinement. Evaluating across six diverse benchmarks, we demonstrate that: (1) stable model semantics allow LLMs to naturally express default rules and exceptions, outperforming SMT-based alternatives by significant margins on nonmonotonic tasks; (2) iterative self-correction is the primary driver of performance, effectively replacing the need for handcrafted domain knowledge; (3) compact in-context reference guides substantially outperform verbose documentation, revealing a "context rot" phenomenon where excessive context hinders constraint adherence.