Constraint-Based Analysis of Reasoning Shortcuts in Neurosymbolic Learning

TL;DR

This paper formalizes reasoning shortcuts in neurosymbolic systems as a constraint satisfaction problem, providing algorithms and complexity results to detect and repair shortcuts, with validation on multiple benchmarks.

Contribution

It introduces an ASP-based method to verify and repair reasoning shortcuts in neurosymbolic learning, along with complexity analysis and sample bounds.

Findings

Verification algorithm is sound and complete.

Repair algorithm converges in at most k iterations.

Deciding shortcut-freeness is coNP-complete.

Abstract

Neurosymbolic systems can satisfy logical constraints during learning without achieving the intended concept-label correspondence; this is a problem known as reasoning shortcuts. We formalize reasoning shortcuts as a constraint satisfaction problem and investigate under which conditions concept mappings are uniquely determined by the constraints. We prove that a discrimination property (requiring that no valid concept mapping can be transformed into another valid mapping by swapping two concept values) is necessary for shortcut-freeness under bijective mappings, but demonstrate via a counterexample that it is insufficient even when the constraint graph is connected. We develop an ASP-based algorithm that verifies whether a given constraint set uniquely determines the intended concept mapping, with proven soundness and completeness. When shortcuts are detected, a greedy repair algorithm eliminates them by augmenting the constraint set, converging in at most $k$ iterations, where $k$ is the number of alternative valid mappings. We further provide a complexity classification: deciding shortcut-freeness is coNP-complete, counting shortcuts is #P-complete, and finding minimal repairs is NP-hard. We also establish sample complexity bounds showing that logarithmically many label queries suffice for disambiguation in favorable cases, while querying all ambiguous positions suffices in the worst case. Experiments across eight benchmark domains validate our approach.