ReSS: Learning Reasoning Models for Tabular Data Prediction via Symbolic Scaffold

TL;DR

ReSS is a framework that combines symbolic decision paths with neural models to improve the accuracy and faithfulness of tabular data reasoning in high-stakes domains.

Contribution

ReSS introduces a novel method to generate grounded natural-language reasoning by integrating symbolic decision paths with LLMs, enhancing interpretability and accuracy.

Findings

ReSS-trained models outperform traditional decision trees and standard fine-tuning by up to 10%.

The framework improves reasoning faithfulness and consistency in medical and financial benchmarks.

Quantitative metrics show reduced hallucination rates and increased explanation sufficiency.

Abstract

Tabular data remains prevalent in high-stakes domains such as healthcare and finance, where predictive models are expected to provide both high accuracy and faithful, human-understandable reasoning. While symbolic models offer verifiable logic, they lack semantic expressiveness. Meanwhile, general-purpose LLMs often require specialized fine-tuning to master domain-specific tabular reasoning. To address the dual challenges of scalable data curation and reasoning consistency, we propose ReSS, a systematic framework that bridges symbolic and neural reasoning models. ReSS leverages a decision-tree model to extract instance-level decision paths as symbolic scaffolds. These scaffolds, alongside input features and labels, guide an LLM to generate grounded natural-language reasoning that strictly adheres to the underlying decision logic. The resulting high-quality dataset is used to fine-tune a pretrained LLM into a specialized tabular reasoning model, further enhanced by a scaffold-invariant data augmentation strategy to improve generalization and explainability. To rigorously assess faithfulness, we introduce quantitative metrics including hallucination rate, explanation necessity, and explanation sufficiency. Experimental results on medical and financial benchmarks demonstrate that ReSS-trained models improve traditional decision trees and standard fine-tuning approaches up to $10\%$ while producing faithful and consistent reasoning