Self-Evolving Recommendation System: End-To-End Autonomous Model Optimization With LLM Agents

TL;DR

This paper introduces a self-evolving recommendation system that uses LLMs to autonomously generate, evaluate, and deploy model improvements, significantly enhancing development speed and model quality in large-scale video recommendation.

Contribution

The paper presents an end-to-end autonomous system leveraging LLMs for hypothesis generation, model optimization, and deployment, reducing manual effort and improving recommendation performance.

Findings

Successful production launches at YouTube demonstrate improved model performance.

Autonomous agents outperform traditional workflows in development speed.

LLM-driven evolution discovers novel optimization and architecture improvements.

Abstract

Optimizing large-scale machine learning systems, such as recommendation models for global video platforms, requires navigating a massive hyperparameter search space and, more critically, designing sophisticated optimizers, architectures, and reward functions to capture nuanced user behaviors. Achieving substantial improvements in these areas is a non-trivial task, traditionally relying on extensive manual iterations to test new hypotheses. We propose a self-evolving system that leverages Large Language Models (LLMs), specifically those from Google's Gemini family, to autonomously generate, train, and deploy high-performing, complex model changes within an end-to-end automated workflow. The self-evolving system is comprised of an Offline Agent (Inner Loop) that performs high-throughput hypothesis generation using proxy metrics, and an Online Agent (Outer Loop) that validates candidates against delayed north star business metrics in live production. Our agents act as specialized Machine Learning Engineers (MLEs): they exhibit deep reasoning capabilities, discovering novel improvements in optimization algorithms and model architecture, and formulating innovative reward functions that target long-term user engagement. The effectiveness of this approach is demonstrated through several successful production launches at YouTube, confirming that autonomous, LLM-driven evolution can surpass traditional engineering workflows in both development velocity and model performance.