STATe-of-Thoughts: Structured Action Templates for Tree-of-Thoughts

TL;DR

STATe introduces a structured, interpretable approach to improve diversity, control, and understanding in large language model reasoning by using high-level action templates instead of stochastic sampling.

Contribution

The paper proposes STATe, a novel method replacing stochastic sampling with discrete, interpretable actions to enhance diversity, control, and interpretability in reasoning with language models.

Findings

STATe achieves greater response diversity than temperature sampling.

Explicit action sequences are highly predictive of output quality.

Estimating action-performance associations guides generation toward promising regions.

Abstract

Inference-Time-Compute (ITC) methods like Best-of-N and Tree-of-Thoughts are meant to produce output candidates that are both high-quality and diverse, but their use of high-temperature sampling often fails to achieve meaningful output diversity. Moreover, existing ITC methods offer limited control over how to perform reasoning, which in turn limits their interpretability. We present STATe Of Thoughts (STATe), an interpretable ITC method that searches over high-level reasoning patterns. STATe replaces stochastic sampling with discrete and interpretable textual interventions: a controller selects actions encoding high-level reasoning choices; a generator produces reasoning steps conditioned on those choices; and an evaluator scores candidates to guide search. This structured approach yields three main advantages. First, action-guided textual interventions reliably influence LLM generations and produce greater response diversity than temperature-based sampling. Second, in a case study on argument generation, STATe's explicit action sequences capture interpretable features that are highly predictive of output quality. Third, estimating the association between performance and action choices allows us to identify promising yet unexplored regions of the action space and steer generation toward them. Together, these results establish STATe as both a practical framework for diverse and controllable text generation, and as a tool for understanding the reasoning patterns that drive performance.