MEMETRON: Metaheuristic Mechanisms for Test-time Response Optimization of Large Language Models

TL;DR

MEMETRON introduces a flexible, model-agnostic framework using metaheuristics to optimize large language model responses at inference time, improving task-specific performance and alignment without retraining.

Contribution

It presents a novel, task-agnostic black-box optimization approach for LLM decoding using hybrid metaheuristics, enhancing response quality and alignment.

Findings

Outperforms standard decoding methods in human preference alignment

Does not require model retraining or gradient access

Generalizes across diverse tasks with lightweight prompts

Abstract

Large language models (LLMs) are increasingly used for both open-ended and structured tasks, yet their inference-time behavior is still largely dictated by heuristic decoding strategies such as greedy search, sampling, or reranking. These methods provide limited control and do not explicitly optimize for task-specific objectives. We introduce MEMETRON, a task-agnostic framework that formulates LLM decoding as a discrete black-box optimization problem. MEMETRON leverages hybrid metaheuristic algorithms, GENETRON and ANNETRON, to search the response space, guided by reward models and contextual operations performed by the LLM itself. This approach enables efficient discovery of high-reward responses without requiring model retraining or gradient access. The framework is modular and generalizes across diverse tasks, requiring only a reward function and lightweight prompt templates. We evaluate our framework on the critical human preference alignment task and demonstrate that it significantly outperforms standard decoding and reranking methods, highlighting its potential to improve alignment without model retraining.