Test-Time Compute for Dense Retrieval: Agentic Program Generation with Frozen Embedding Models

TL;DR

This paper demonstrates that test-time compute can significantly improve small frozen embedding models in dense retrieval tasks through an agentic program-search approach.

Contribution

It introduces a novel agentic program-search method that enhances small embedding models without retraining, achieving significant retrieval performance gains.

Findings

Test-time compute improves small embedding models' performance.

The Pareto frontier collapses onto a simple algebraic default.

Statistically significant nDCG@10 improvements across multiple models.

Abstract

Test-time compute is widely believed to benefit only large reasoning models. We show it also helps small embedding models. Since modern embedding models are distilled from LLM backbones, a frozen encoder should benefit from extra inference compute without retraining. Using an agentic program-search loop, we explore 259 candidate inference programs over a frozen embedding API across ninety generations. The entire Pareto frontier collapses onto a single algebra: a softmax-weighted centroid of the local top-K documents interpolated with the query. This default, which introduces no trainable parameters, lifts nDCG@10 statistically significantly across seven embedding-model families spanning a tenfold parameter range, with held-out full-BEIR validation confirming the lift on every model tested.