The Shape of Reasoning: Topological Analysis of Reasoning Traces in Large Language Models

TL;DR

This paper presents a topological data analysis framework for evaluating reasoning traces in large language models, outperforming traditional graph metrics in predicting reasoning quality.

Contribution

It introduces a novel TDA-based evaluation method that captures the geometry of reasoning traces, enabling more effective and label-efficient assessment.

Findings

Topological features outperform standard graph metrics in predicting reasoning quality.

A stable set of topological features reliably indicates trace quality.

The approach offers a practical signal for reinforcement learning algorithms.

Abstract

Evaluating the quality of reasoning traces from large language models remains understudied, labor-intensive, and unreliable: current practice relies on expert rubrics, manual annotation, and slow pairwise judgments. Automated efforts are dominated by graph-based proxies that quantify structural connectivity but do not clarify what constitutes high-quality reasoning; such abstractions can be overly simplistic for inherently complex processes. We introduce a topological data analysis (TDA)-based evaluation framework that captures the geometry of reasoning traces and enables label-efficient, automated assessment. In our empirical study, topological features yield substantially higher predictive power for assessing reasoning quality than standard graph metrics, suggesting that effective reasoning is better captured by higher-dimensional geometric structures rather than purely relational graphs. We further show that a compact, stable set of topological features reliably indicates trace quality, offering a practical signal for future reinforcement learning algorithms.