Test-Time Scaling with Repeated Sampling Improves Multilingual Text Generation

TL;DR

This paper demonstrates that test-time scaling with repeated sampling enhances multilingual text generation quality, especially when using reward-based verifiers for reasoning tasks, across multiple benchmarks.

Contribution

It introduces a novel evaluation of repeated sampling at inference time for multilingual generation, highlighting the importance of verifier selection for different task types.

Findings

Repeated sampling improves multilingual generation quality by over 35%.

Reward-based verifiers outperform perplexity-based scoring on reasoning tasks.

Perplexity scoring is effective for open-ended prompts.

Abstract

Inference-time scaling via repeated sampling has shown promise in reasoning tasks, but its effectiveness in multilingual generation remains underexplored. We evaluate this approach using perplexity- and reward-based verifiers on two multilingual benchmarks: the Aya Evaluation Suite and m-ArenaHard. Our results show consistent quality improvements, with gains exceeding 35% in some cases. While perplexity-based scoring is effective for open-ended prompts, only reward-based verifiers improve performance on tasks requiring reasoning (e.g., math, code). Our results demonstrate the broader utility of repeated sampling for multilingual text generation and underscore the importance of selecting right verifiers for the task.