DTS: Enhancing Large Reasoning Models via Decoding Tree Sketching

TL;DR

DTS is a decoding framework that improves large reasoning models by exploring reasoning trees efficiently and selecting high-quality solutions, leading to significant accuracy gains and reduced redundancy.

Contribution

We introduce Decoding Tree Sketching (DTS), a novel plug-and-play method for structural multi-trajectory exploration and reasoning selection in large reasoning models.

Findings

Improves accuracy by 14% across multiple models and datasets.

Reduces repetitive generation by 8% on average.

Enables smaller models to outperform larger ones by 10x size.

Abstract

Large Reasoning Models (LRMs) achieve remarkable inference-time improvements through parallel thinking. However, existing methods rely on redundant sampling of reasoning trajectories, failing to effectively explore the reasoning space to uncover high-quality solutions. To address these limitations, we propose Decoding Tree Sketching (DTS), a plug-and-play decoding framework for structural multi-trajectory exploration and reasoning selection. For reasoning exploration, DTS sketches a backbone tree of the reasoning space by selectively branching at decision tokens. For reasoning selection, guided by length-accuracy anti-correlation, DTS designs an early termination to prioritize short and reliable trajectories during decoding. Experimental results across four LRMs and datasets demonstrate that DTS significantly enhances accuracy by 14% and reduces repetitive generation by 8% on average. Notably, DTS enables smaller models to outperform larger models with 10$\times$ the size, highlighting its potential to strengthen reasoning capabilities.