Reasoning Path Divergence: A New Metric and Curation Strategy to Unlock LLM Diverse Thinking

TL;DR

This paper introduces a new training paradigm and metric to enhance the diversity of reasoning paths in large language models, leading to improved reasoning performance and output variety.

Contribution

It proposes the 1PNS training paradigm and Reasoning Path Divergence metric to increase inference diversity and improve reasoning accuracy in LLMs.

Findings

RPD-selected training increases output diversity

Achieves +2.80% pass@16 improvement over baseline

Enhances reasoning performance on AIME24 dataset

Abstract

While Test-Time Scaling (TTS) has proven effective in improving the reasoning ability of large language models (LLMs), low diversity in model outputs often becomes a bottleneck; this is partly caused by the common "one problem, one solution" (1P1S) training practice, which provides a single canonical answer and can push models toward a narrow set of reasoning paths. This homogenization not only limits sampling effectiveness but also restricts the exploration space for subsequent Reinforcement Learning (RL) stages. To address this, we propose a "one problem, multiple solutions" (1PNS) training paradigm that exposes the model to a variety of valid reasoning trajectories and thus increases inference diversity. A core challenge for 1PNS is reliably measuring semantic differences between multi-step chains of thought, so we introduce Reasoning Path Divergence (RPD), a step-level metric that aligns and scores Long Chain-of-Thought solutions to capture differences in intermediate reasoning. Using RPD, we curate maximally diverse solution sets per problem and fine-tune Qwen3-4B-Base. Experiments show that RPD-selected training yields more varied outputs and higher pass@k, with an average +2.80% gain in pass@16 over a strong 1P1S baseline and a +4.99% gain on AIME24, demonstrating that 1PNS further amplifies the effectiveness of TTS. Our code is available at https://github.com/fengjujf/Reasoning-Path-Divergence .