Certified Self-Consistency: Statistical Guarantees and Test-Time Training for Reliable Reasoning in LLMs

TL;DR

This paper introduces a unified statistical framework for certifiable inference in large language models, connecting self-consistency and test-time reinforcement learning, and providing guarantees for reliable reasoning without extra supervision.

Contribution

It develops a certifiable inference framework, introduces the Martingale Majority Certificate, and links self-consistency with TTRL under a common statistical perspective.

Findings

Majority voting provides a high-probability certificate of self-consistency.

Finite-sample and anytime-valid bounds quantify confidence levels.

Post-training methods like TTRL sharpen answer distributions, reducing certification samples.

Abstract

Recent advances such as self-consistency and test-time reinforcement learning (TTRL) improve the reliability of large language models (LLMs) without additional supervision, yet their underlying mechanisms and statistical guarantees remain poorly understood. We present a unified framework for certifiable inference in LLMs, showing that majority voting provides a statistical certificate of self-consistency: under mild assumptions, the aggregated answer coincides with the mode of the model's terminal distribution with high probability. We derive finite-sample and anytime-valid concentration bounds that quantify this confidence, and introduce the Martingale Majority Certificate (MMC), a sequential stopping rule that adaptively determines when sufficient samples have been drawn. We further prove that label-free post-training methods such as TTRL implicitly sharpen the answer distribution by exponentially tilting it toward its mode, thereby reducing the number of samples required for certification. Building on this insight, we propose new post-training objectives that explicitly optimise this trade-off between sharpness and bias. Together, these results explain and connect two central test-time scaling strategies, self-consistency and TTRL, within a single statistical framework for label-free, certifiable reliability in reasoning LLMs.