MLZero: A Multi-Agent System for End-to-end Machine Learning Automation

TL;DR

MLZero is a multi-agent system utilizing LLMs for fully automated end-to-end machine learning across various data types, reducing manual effort and outperforming existing AutoML solutions.

Contribution

We introduce MLZero, a novel multi-agent framework with enhanced memory and perception modules that automates ML workflows across multimodal data with minimal human input.

Findings

Outperforms all competitors on MLE-Bench Lite with six gold medals.

Achieves a success rate of 0.92 on the Multimodal AutoML Agent Benchmark, a 263.6% improvement.

Effective even with an 8B LLM, surpassing larger existing systems.

Abstract

Existing AutoML systems have advanced the automation of machine learning (ML); however, they still require substantial manual configuration and expert input, particularly when handling multimodal data. We introduce MLZero, a novel multi-agent framework powered by Large Language Models (LLMs) that enables end-to-end ML automation across diverse data modalities with minimal human intervention. A cognitive perception module is first employed, transforming raw multimodal inputs into perceptual context that effectively guides the subsequent workflow. To address key limitations of LLMs, such as hallucinated code generation and outdated API knowledge, we enhance the iterative code generation process with semantic and episodic memory. MLZero demonstrates superior performance on MLE-Bench Lite, outperforming all competitors in both success rate and solution quality, securing six gold medals. Additionally, when evaluated on our Multimodal AutoML Agent Benchmark, which includes 25 more challenging tasks spanning diverse data modalities, MLZero outperforms the competing methods by a large margin with a success rate of 0.92 (+263.6\%) and an average rank of 2.28. Our approach maintains its robust effectiveness even with a compact 8B LLM, outperforming full-size systems from existing solutions.