ReTrace: Interactive Visualizations for Reasoning Traces of Large Reasoning Models

TL;DR

ReTrace is an interactive visualization system designed to structure and display reasoning traces from large language models, enhancing user understanding and interpretability of complex AI reasoning processes.

Contribution

This work introduces ReTrace, a novel interactive visualization tool that structures reasoning traces using a validated taxonomy to improve comprehension of large model reasoning.

Findings

Visualizations improved understanding accuracy

Users perceived less effort with visualizations

Study demonstrated enhanced interpretability of reasoning traces

Abstract

Recent advances in Large Language Models have led to Large Reasoning Models, which produce step-by-step reasoning traces. These traces offer insight into how models think and their goals, improving explainability and helping users follow the logic, learn the process, and even debug errors. These traces, however, are often verbose and complex, making them cognitively demanding to comprehend. We address this challenge with ReTrace, an interactive system that structures and visualizes textual reasoning traces to support understanding. We use a validated reasoning taxonomy to produce structured reasoning data and investigate two types of interactive visualizations thereof. In a controlled user study, both visualizations enabled users to comprehend the model's reasoning more accurately and with less perceived effort than a raw text baseline. The results of this study could have design implications for making long and complex machine-generated reasoning processes more usable and transparent, an important step in AI explainability.