SEO: Stochastic Experience Optimization for Large Language Models

TL;DR

This paper introduces Stochastic Experience Optimization (SEO), a novel iterative method that enhances large language models by finding optimized, model-specific experiences through natural language updates, leading to improved performance and generalization.

Contribution

The paper presents SEO, a new approach that optimizes experiences for LLMs without altering model parameters, using stochastic validation to ensure effective updates.

Findings

SEO-optimized experiences improve LLM performance across tasks

SEO experiences generalize well to out-of-distribution data

The method achieves consistent performance gains in experiments

Abstract

Large Language Models (LLMs) can benefit from useful experiences to improve their performance on specific tasks. However, finding helpful experiences for different LLMs is not obvious, since it is unclear what experiences suit specific LLMs. Previous studies intended to automatically find useful experiences using LLMs, while it is difficult to ensure the effectiveness of the obtained experience. In this paper, we propose Stochastic Experience Optimization (SEO), an iterative approach that finds optimized model-specific experience without modifying model parameters through experience update in natural language. In SEO, we propose a stochastic validation method to ensure the update direction of experience, avoiding unavailing updates. Experimental results on three tasks for three LLMs demonstrate that experiences optimized by SEO can achieve consistently improved performance. Further analysis indicates that SEO-optimized experience can generalize to out-of-distribution data, boosting the performance of LLMs on similar tasks.