Feedback Descent: Open-Ended Text Optimization via Pairwise Comparison

TL;DR

Feedback Descent introduces a novel text optimization framework that leverages detailed structured feedback instead of scalar rewards, enabling more effective and directed optimization of prompts, code, and molecules without modifying model weights.

Contribution

It presents a new method for text artifact optimization using structured textual feedback and in-context learning, outperforming existing methods across multiple domains.

Findings

Outperforms state-of-the-art prompt optimization methods.

Achieves superior results in molecular discovery benchmarks.

Enables targeted edits through in-context learning with structured feedback.

Abstract

We introduce \textit{Feedback Descent}, a framework that optimizes text artifacts -- prompts, code, and molecules -- through structured textual feedback, rather than relying solely on scalar rewards. By preserving detailed critiques instead of compressing them to binary preferences, Feedback Descent widens the information bottleneck in preference learning, enabling directed optimization in text space rather than weight space. We show that in-context learning can transform structured feedback into gradient-like directional information, enabling targeted edits. Unlike prior approaches that collapse judgments into single bits, our evaluators pair each comparison with textual feedback, which functions as high-bandwidth supervision. The iteration loop is done purely at inference time, without modifying any model weights, and is task-agnostic. We evaluate Feedback Descent on three diverse domains and find that it outperforms state-of-the-art prompt optimization (GEPA), reinforcement learning methods (GRPO, REINVENT), and even specialized graph-based molecular optimizers. In the DOCKSTRING molecule discovery benchmark, Feedback Descent identifies novel drug-like molecules surpassing the $99.9$th percentile of a database with more than $260{,}000$ compounds across six protein targets.