Structured Decomposition for LLM Reasoning: Cross-Domain Validation and Semantic Web Integration

TL;DR

This paper introduces a hybrid reasoning framework combining LLMs and symbolic reasoners, validated across multiple domains, to improve rule application accuracy and interpretability in structured decision-making tasks.

Contribution

It presents a novel integration pattern that uses LLMs for ontology population and symbolic reasoners for rule enforcement, enhancing interpretability and formal guarantees in rule-based reasoning.

Findings

Structured decomposition outperforms few-shot prompting in accuracy

Symbolic verification significantly improves reasoning reliability

Framework is validated across legal, scientific, and clinical domains

Abstract

Rule-based reasoning over natural language input arises in domains where decisions must be auditable and justifiable: clinical protocols specify eligibility criteria in prose, evidence rules define admissibility through textual conditions, and scientific standards dictate methodological requirements. Applying rules to such inputs demands both interpretive flexibility and formal guarantees. Large language models (LLMs) provide flexibility but cannot ensure consistent rule application; symbolic systems provide guarantees but require structured input. This paper presents an integration pattern that combines these strengths: LLMs serve as ontology population engines, translating unstructured text into ABox assertions according to expert-authored TBox specifications, while SWRL-based reasoners apply rules with deterministic guarantees. The framework decomposes reasoning into entity identification, assertion extraction, and symbolic verification, with task definitions grounded in OWL 2 ontologies. Experiments across three domains (legal hearsay determination, scientific method-task application, clinical trial eligibility) and eleven language models validate the approach. Structured decomposition achieves statistically significant improvements over few-shot prompting in aggregate, with gains observed across all three domains. An ablation study confirms that symbolic verification provides substantial benefit beyond structured prompting alone. The populated ABox integrates with standard semantic web tooling for inspection and querying, positioning the framework for richer inference patterns that simpler formalisms cannot express.