Flipping Against All Odds: Reducing LLM Coin Flip Bias via Verbalized Rejection Sampling

TL;DR

This paper introduces Verbalized Rejection Sampling (VRS), a method that uses natural language prompts to reduce bias in LLM-generated Bernoulli samples, improving their reliability for stochastic tasks.

Contribution

The paper presents VRS, a novel natural-language adaptation of rejection sampling that enhances LLM sampling fidelity without complex engineering or internal access.

Findings

VRS significantly reduces sampling bias across different LLMs.

Theoretical analysis confirms VRS's effectiveness under mild assumptions.

VRS leverages classical probabilistic tools embedded in natural language prompts.

Abstract

Large language models (LLMs) can often accurately describe probability distributions using natural language, yet they still struggle to generate faithful samples from them. This mismatch limits their use in tasks requiring reliable stochasticity, such as Monte Carlo methods, agent-based simulations, and randomized decision-making. We investigate this gap between knowledge and sampling in the context of Bernoulli distributions. We introduce Verbalized Rejection Sampling (VRS), a natural-language adaptation of classical rejection sampling that prompts the LLM to reason about and accept or reject proposed samples. Despite relying on the same Bernoulli mechanism internally, VRS substantially reduces sampling bias across models. We provide theoretical analysis showing that, under mild assumptions, VRS improves over direct sampling, with gains attributable to both the algorithm and prompt design. More broadly, our results show how classical probabilistic tools can be verbalized and embedded into LLM workflows to improve reliability, without requiring access to model internals or heavy prompt engineering.