Output-Space Search: Targeting LLM Generations in a Frozen Encoder-Defined Output Space

TL;DR

This paper presents Output-Space Search (OS-Search), a novel method that transforms language model generation into an endpoint search in a frozen encoder-defined output space, enabling efficient optimization and diversity enhancement.

Contribution

OS-Search introduces a new framework for LLM generation by performing endpoint search in a fixed output space, allowing parallel optimization and improved diversity and quality.

Findings

Achieves 3.1x higher diversity on stories compared to prompt-chaining.

Improves objective scores in code generation while maintaining validity.

Enables black-box optimization without path-dependent token search.

Abstract

We introduce Output-Space Search (OS-Search), which turns LLM generation into endpoint search. An outer loop selects a target z* in a frozen encoder-defined 3D output space Z, and a retrieval-grounded policy trained with sequence-level RL generates outputs whose coordinates land near z* under standard autoregressive decoding. This enables parallel sweeps and black-box optimization in Z without path-dependent token/program search. On stories, sweeping Z (text) yields 3.1x higher LLM-scored diversity than prompt-chaining. On code, Bayesian optimization over Z (code) improves an objective withheld from the controller under matched inference budgets while preserving validity.