RankSteer: Activation Steering for Pointwise LLM Ranking

TL;DR

RankSteer introduces a post-hoc activation steering method that enhances zero-shot pointwise LLM ranking by controlling internal representation directions, improving ranking quality without retraining or explicit cross-document comparisons.

Contribution

This work presents a novel activation steering framework for LLM ranking, identifying and manipulating three key representation directions to improve zero-shot ranking performance.

Findings

Consistently improves ranking quality on TREC DL 20 and BEIR benchmarks.

Requires only a small number of anchor queries for calibration.

Provides geometric insights into LLM relevance representation.

Abstract

Large language models (LLMs) have recently shown strong performance as zero-shot rankers, yet their effectiveness is highly sensitive to prompt formulation, particularly role-play instructions. Prior analyses suggest that role-related signals are encoded along activation channels that are largely separate from query-document representations, raising the possibility of steering ranking behavior directly at the activation level rather than through brittle prompt engineering. In this work, we propose RankSteer, a post-hoc activation steering framework for zero-shot pointwise LLM ranking. We characterize ranking behavior through three disentangled and steerable directions in representation space: a \textbf{decision direction} that maps hidden states to relevance scores, an \textbf{evidence direction} that captures relevance signals not directly exploited by the decision head, and a \textbf{role direction} that modulates model behavior without injecting relevance information. Using projection-based interventions at inference time, RankSteer jointly controls these directions to calibrate ranking behavior without modifying model weights or introducing explicit cross-document comparisons. Experiments on TREC DL 20 and multiple BEIR benchmarks show that RankSteer consistently improves ranking quality using only a small number of anchor queries, demonstrating that substantial ranking capacity remains under-utilized in pointwise LLM rankers. We further provide a geometric analysis revealing that steering improves ranking by stabilizing ranking geometry and reducing dispersion, offering new insight into how LLMs internally represent and calibrate relevance judgments.