Dual-Track CoT: Budget-Aware Stepwise Guidance for Small LMs
Sagnik Chatterjee, Atharva Patil, Sricharan Ramesh

TL;DR
This paper introduces Dual-Track CoT, a method that enhances small language models' reasoning capabilities by using budget-aware, stepwise guidance, enabling reliable multi-step reasoning under limited tokens and compute.
Contribution
It proposes a novel approach that improves small language models' reasoning efficiency and accuracy through fine-grained, budget-aware stepwise guidance, reducing reliance on large models or extensive sampling.
Findings
Dual-Track CoT improves reasoning accuracy of small LMs under token constraints.
The method achieves better performance with fewer tokens compared to existing approaches.
It demonstrates practical benefits for low-resource or on-device reasoning tasks.
Abstract
Large Language Models (LLMs) solve many reasoning tasks via chain-of-thought (CoT) prompting, but smaller models (about 7 to 8B parameters) still struggle with multi-step reasoning under tight compute and token budgets. Existing test time reasoning methods such as self consistency (sampling multiple rationales and voting), Tree-of-Thoughts (search over intermediate thoughts), and critique revise loops improve performance, but often at high token cost and without fine-grained step-level control. This project1 aims to address that gap: can Small Language Models (SLMs) reason reliably using the same or fewer tokens? This question is both scientific and practical. Scientifically, it probes whether process supervision and simple test-time controls (such as token budgets and rejection of redundant steps) can substitute for model scale or large sampling counts. Practically, many deployments…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
