Test-time Scaling Techniques in Theoretical Physics -- A Comparison of Methods on the TPBench Dataset

TL;DR

This paper evaluates and compares test-time scaling methods for large language models in advanced theoretical physics, introducing a symbolic weak-verifier framework that improves performance on physics and mathematical reasoning benchmarks.

Contribution

It introduces a novel symbolic weak-verifier framework that enhances test-time scaling effectiveness in physics reasoning tasks, demonstrating superior performance on TPBench and AIME datasets.

Findings

The symbolic weak-verifier significantly outperforms existing methods on TPBench.

Test-time scaling methods can generalize from mathematical to physics reasoning.

The proposed approach improves parallel scaling results in complex scientific problems.

Abstract

Large language models (LLMs) have shown strong capabilities in complex reasoning, and test-time scaling techniques can enhance their performance with comparably low cost. Many of these methods have been developed and evaluated on mathematical reasoning benchmarks such as AIME. This paper investigates whether the lessons learned from these benchmarks generalize to the domain of advanced theoretical physics. We evaluate a range of common test-time scaling methods on the TPBench physics dataset and compare their effectiveness with results on AIME. To better leverage the structure of physics problems, we develop a novel, symbolic weak-verifier framework to improve parallel scaling results. Our empirical results demonstrate that this method significantly outperforms existing test-time scaling approaches on TPBench. We also evaluate our method on AIME, confirming its effectiveness in solving advanced mathematical problems. Our findings highlight the power of step-wise symbolic verification for tackling complex scientific problems.