NEX: Neuron Explore-Exploit Scoring for Label-Free Chain-of-Thought Selection and Model Ranking

TL;DR

NEX introduces an unsupervised, neuron-based scoring method for selecting and ranking large language model outputs during inference, reducing reliance on labeled data and improving efficiency.

Contribution

The paper presents NEX, a novel label-free framework that uses neuron activation patterns to identify exploration and exploitation phases, enabling effective response ranking without supervision.

Findings

NEX accurately predicts downstream accuracy across benchmarks.

Neuron activation spikes correlate with reasoning exploration phases.

NEX outperforms existing methods in model response selection.

Abstract

Large language models increasingly spend inference compute sampling multiple chain-of-thought traces or searching over merged checkpoints. This shifts the bottleneck from generation to selection, often without supervision on the target distribution. We show entropy-based exploration proxies follow an inverted-U with accuracy, suggesting extra exploration can become redundant and induce overthinking. We propose NEX, a white-box label-free unsupervised scoring framework that views reasoning as alternating E-phase (exploration) and X-phase (exploitation). NEX detects E-phase as spikes in newly activated MLP neurons per token from sparse activation caches, then uses a sticky two-state HMM to infer E-X phases and credits E-introduced neurons by whether they are reused in the following X span. These signals yield interpretable neuron weights and a single Good-Mass Fraction score to rank candidate responses and merged variants without task answers. Across reasoning benchmarks and Qwen3 merge families, NEX computed on a small unlabeled activation set predicts downstream accuracy and identifies better variants; we further validate the E-X signal with human annotations and provide causal evidence via "Effective-vs-Redundant" neuron transfer.