Temporalizing Confidence: Evaluation of Chain-of-Thought Reasoning with Signal Temporal Logic

TL;DR

This paper introduces a novel framework that models and evaluates the confidence of chain-of-thought reasoning in large language models using Signal Temporal Logic, improving the reliability of confidence estimates.

Contribution

It proposes a formal STL-based method to evaluate and reshape confidence signals in LLM reasoning, enhancing interpretability and calibration.

Findings

Improves confidence calibration metrics

Provides more reliable uncertainty estimates

Enforces smoothness and causal consistency

Abstract

Large Language Models (LLMs) have shown impressive performance in mathematical reasoning tasks when guided by Chain-of-Thought (CoT) prompting. However, they tend to produce highly confident yet incorrect outputs, which poses significant risks in domains like education, where users may lack the expertise to assess reasoning steps. To address this, we propose a structured framework that models stepwise confidence as a temporal signal and evaluates it using Signal Temporal Logic (STL). In particular, we define formal STL-based constraints to capture desirable temporal properties and compute robustness scores that serve as structured, interpretable confidence estimates. Our approach also introduces a set of uncertainty reshaping strategies to enforce smoothness, monotonicity, and causal consistency across the reasoning trajectory. Experiments show that our approach consistently improves calibration metrics and provides more reliable uncertainty estimates than conventional confidence aggregation and post-hoc calibration.