GenSelect: A Generative Approach to Best-of-N

TL;DR

GenSelect is a new method that uses large language models' reasoning abilities to select the best solution among many, improving efficiency and performance in reasoning tasks.

Contribution

It introduces a generative approach that leverages LLMs' comparative reasoning to select solutions, outperforming existing scoring methods in reasoning tasks.

Findings

GenSelect outperforms pointwise and pairwise scoring methods.

It scales efficiently with larger sampling budgets.

It improves accuracy in math reasoning tasks.

Abstract

Generative reward models with parallel sampling have enabled effective test-time scaling for reasoning tasks. Current approaches employ pointwise scoring of individual solutions or pairwise comparisons. However, pointwise methods underutilize LLMs' comparative abilities, while pairwise methods scale inefficiently with larger sampling budgets. We introduce GenSelect, where the LLM uses long reasoning to select the best solution among N candidates. This leverages LLMs' comparative strengths while scaling efficiently across parallel sampling budgets. For math reasoning, we demonstrate that reasoning models, such as QwQ and DeepSeek-R1-0528, excel at GenSelect, outperforming existing scoring approaches with simple prompting.