Exploring Diverse Generation Paths via Inference-time Stiefel Activation Steering

TL;DR

This paper introduces STARS, a training-free, inference-time method that enhances diversity in language model outputs by optimizing activation steering directions on the Stiefel manifold, promoting divergent generation paths.

Contribution

The paper proposes a novel, manifold-based activation steering technique for inference-time diversity promotion in language models, without additional training.

Findings

STARS outperforms standard sampling methods in diversity metrics.

It achieves greater output diversity without losing qualitative quality.

The method is efficient enough for real-time inference.

Abstract

Language models often default to a narrow set of high-probability outputs, leaving their generation paths homogeneous and prone to mode collapse. Sampling-based strategies inject randomness but still struggle to guarantee diversity across multiple concurrent generation runs. We address this limitation by introducing STARS ($\textbf{St}$iefel-based $\textbf{A}$ctivation Steering for Diverse $\textbf{R}$ea$\textbf{S}$oning), a training-free, inference-time intervention method that transforms activation steering into an exploration engine. At each token, STARS collects the hidden activations of concurrent generation runs and optimizes multiple additive steering directions jointly on the Stiefel manifold. STARS maximizes the geometric volume of the steered activations, while the Stiefel manifold induces orthogonality of the steering interventions. This formulation explicitly promotes divergent activation vectors of concurrent generation runs, and implicitly promotes divergent generation trajectories. This manifold optimization formulation can be solved using a Riemannian gradient descent algorithm with convergence guarantees, but this algorithm is too time-consuming for real-time inference. To guarantee low latency, we further design a lightweight one-step update with an aggressive, closed-form stepsize. For test case generation and scientific discovery benchmarks, STARS consistently outperforms standard sampling methods, achieving greater diversity without sacrificing qualitative performance.