TREX: Automating LLM Fine-tuning via Agent-Driven Tree-based Exploration

TL;DR

TREX is a multi-agent system that automates the entire LLM training process, from research to evaluation, using a tree-based exploration approach to optimize model performance.

Contribution

Introducing TREX, the first system to automate LLM training lifecycle with multi-agent collaboration and search tree modeling for efficient exploration.

Findings

TREX effectively automates LLM training workflows.

The system outperforms baseline methods in optimizing model performance.

FT-Bench provides a new benchmark for evaluating automated LLM training.

Abstract

While Large Language Models (LLMs) have empowered AI research agents to perform isolated scientific tasks, automating complex, real-world workflows, such as LLM training, remains a significant challenge. In this paper, we introduce TREX, a multi-agent system that automates the entire LLM training life-cycle. By orchestrating collaboration between two core modules-the Researcher and the Executor-the system seamlessly performs requirement analysis, open-domain literature and data research, formulation of training strategies, preparation of data recipes, and model training and evaluation. The multi-round experimental process is modeled as a search tree, enabling the system to efficiently plan exploration paths, reuse historical results, and distill high-level insights from iterative trials. To evaluate the capability of automated LLM training, we construct FT-Bench, a benchmark comprising 10 tasks derived from real-world scenarios, ranging from optimizing fundamental model capabilities to enhancing performance on domain-specific tasks. Experimental results demonstrate that the TREX agent consistently optimizes model performance on target tasks.