On the Effect of Sampling Diversity in Scaling LLM Inference

TL;DR

This paper systematically analyzes how prompt sampling diversity influences large language model inference, providing theoretical insights and empirical results that demonstrate significant performance gains when diversity is effectively utilized.

Contribution

It offers a theoretical explanation for the benefits of diversified sampling in LLM inference and introduces a diversity-fidelity trade-off principle for designing sampling strategies.

Findings

Diversified sampling reduces error rates in Best-of-N inference.

Effective diversity strategies can improve reasoning, mathematics, and code generation performance.

Diversity may diminish under majority voting, affecting inference outcomes.

Abstract

Large language model (LLM) scaling inference is key to unlocking greater performance, and leveraging diversity has proven an effective way to enhance it. Motivated by the observed relationship between solution accuracy and meaningful response diversity, we systematically study the effect of prompt diversity in scaling inference. We theoretically explain why diversified sampling improves Best-of-N scaling, showing that responses generated from diverse prompts after Best-of-N selection exhibit significantly lower error rates than those produced from stationary prompts. Building on this analysis, we derive a diversity-fidelity trade-off principle, that guides the design of sampling strategies introducing diversity. From this guidance, we instantiate a family of effective perturbation styles. We theoretically and empirically characterize when diversified exploration remains effective, demonstrating that it works under a variety of conditions, and we further show that under majority voting, diversity may vanish. Finally, we systematically evaluate how effective sampling diversity is and show that, when applied appropriately in different contexts, it yields relative gains of 10.8% in EM@100 for reasoning, 9.6% for mathematics, and 9.5% in Pass@100 for code generation. Overall, this work provides a systematic analysis that offers a theoretical and empirical foundation for understanding how sampling diversity affects LLM inference-time scaling.