The Blessing of Dimensionality in LLM Fine-tuning: A Variance-Curvature Perspective

TL;DR

This paper reveals that the low-dimensional curvature structure of fine-tuning landscapes allows small population evolution strategies to effectively optimize large language models, explaining observed non-monotonic training behaviors.

Contribution

It introduces a geometric perspective on fine-tuning landscapes, demonstrating that high-curvature directions dominate and enable efficient optimization with small populations.

Findings

Small populations can effectively fine-tune billion-parameter models.

Reward improvements often rise and then degrade due to landscape geometry.

High-curvature directions dominate the fine-tuning landscape.

Abstract

Weight-perturbation evolution strategies (ES) can fine-tune billion-parameter language models with surprisingly small populations (e.g., $N\!\approx\!30$), contradicting classical zeroth-order curse-of-dimensionality intuition. We also observe a second seemingly separate phenomenon: under fixed hyperparameters, the stochastic fine-tuning reward often rises, peaks, and then degrades in both ES and GRPO. We argue that both effects reflect a shared geometric property of fine-tuning landscapes: they are low-dimensional in curvature. A small set of high-curvature dimensions dominates improvement, producing (i) heterogeneous time scales that yield rise-then-decay under fixed stochasticity, as captured by a minimal quadratic stochastic-ascent model, and (ii) degenerate improving updates, where many random perturbations share similar components along these directions. Using ES as a geometric probe on fine-tuning reward landscapes of GSM8K, ARC-C, and WinoGrande across Qwen2.5-Instruct models (0.5B--7B), we show that reward-improving perturbations remain empirically accessible with small populations across scales. Together, these results reconcile ES scalability with non-monotonic training dynamics and suggest that high-dimensional fine-tuning may admit a broader class of viable optimization methods than worst-case theory implies.