Procedural Refinement by LLM-driven Algorithmic Debugging for ARC-AGI-2

TL;DR

This paper introduces ABPR, a neuro-symbolic refinement method combining LLMs with Prolog for semantic rule re-checking, significantly improving abstract reasoning performance on benchmarks like ARC-AGI-2.

Contribution

It presents a novel abduction-based refinement approach that enhances LLM-based rule induction through formal program debugging techniques.

Findings

ABPR achieves 56.67% Pass@2 on ARC-AGI-2.

With GPT-5.5 xHigh, ABPR reaches 98.33% Pass@2.

The framework extends to relational and analogical abstraction tasks.

Abstract

In high-complexity abstract reasoning, a system must infer a latent rule from a few examples or structured observations and apply it to unseen instances. LLMs can express such rules as programs, but ordinary conversation-based refinement is largely outcome-level: it observes that an answer or output is wrong without formally re-checking which abstraction, relation, or transformation justified that outcome. We propose \emph{Abduction-Based Procedural Refinement} (ABPR), a neuro-symbolic refinement approach that couples an LLM with a Prolog meta-interpreter. ABPR treats each candidate program as an executable declarative hypothesis of the latent rule and reifies its SLD goal--subgoal resolution into compact proof-tree-style derivations, following Shapiro's algorithmic program debugging (APD). In this view, refinement is not merely code-level debugging, but semantic re-checking of the model's hypothesised rule. We evaluate ABPR primarily on ARC-AGI-2, a challenging few-shot abstract rule induction benchmark over grid transformations. ABPR with Gemini-3-Flash achieves 56.67\% Pass@2, while GPT-5.5 xHigh with ABPR reaches 98.33\% Pass@2 on the public evaluation set. Supplementary experiments on fill-in-the-blank I-RAVEN-X and A-I-RAVEN adaptations provide evidence that the same trace-guided framework extends beyond ARC-specific grid tasks to RAVEN-style relational and analogical abstraction. Repeated-run and sensitivity analyses show that parallel trace-guided search reduces stochastic variance as search breadth and refinement depth increase.