Divide, Optimize, Merge: Fine-Grained LLM Agent Optimization at Scale

TL;DR

This paper introduces Fine-Grained Optimization (FGO), a scalable framework for optimizing large language model-based agents by dividing tasks, targeted optimization, and progressive merging, which improves performance and reduces token usage.

Contribution

The paper presents FGO, a novel scalable framework for LLM agent optimization that overcomes context limitations and enhances efficiency compared to existing methods.

Findings

FGO outperforms existing approaches by 1.6-8.6% in benchmarks.

FGO reduces prompt token consumption by 56.3%.

FGO achieves consistent performance gains across various dataset sizes.

Abstract

LLM-based optimization has shown remarkable potential in enhancing agentic systems. However, the conventional approach of prompting LLM optimizer with the whole training trajectories on training dataset in a single pass becomes untenable as datasets grow, leading to context window overflow and degraded pattern recognition. To address these challenges, we propose Fine-Grained Optimization (FGO), a scalable framework that divides large optimization tasks into manageable subsets, performs targeted optimizations, and systematically combines optimized components through progressive merging. Evaluation across ALFWorld, LogisticsQA, and GAIA benchmarks demonstrate that FGO outperforms existing approaches by 1.6-8.6% while reducing average prompt token consumption by 56.3%. Our framework provides a practical solution for scaling up LLM-based optimization of increasingly sophisticated agent systems. Further analysis demonstrates that FGO achieves the most consistent performance gain in all training dataset sizes, showcasing its scalability and efficiency.