MLR-Copilot: Autonomous Machine Learning Research based on Large Language Models Agents

TL;DR

MLR-COPILOT is an autonomous framework that uses large language models to generate, implement, and execute machine learning research ideas, aiming to accelerate ML research productivity and innovation.

Contribution

This work introduces a novel LLM-powered autonomous research framework with a three-stage process for idea generation, experiment implementation, and execution, enhancing ML research automation.

Findings

Successfully generated feasible research ideas from existing papers.

Automated implementation of experiments with retrieved code and models.

Demonstrated potential to accelerate ML research progress.

Abstract

Autonomous machine learning research has gained significant attention recently. We present MLR-COPILOT, an autonomous Machine Learning Research framework powered by large language model agents. The system is designed to enhance ML research productivity through automatic generation and implementation of research ideas within constraints. Our work was released in August 2024 (concurrent to AI-Scientist) and has gained notable recognition from leading projects. We further enhance our ideation with training afterwards. The framework consists of three stages: idea generation, experiment implementation, and code execution. First, existing research papers are used to generate feasible ideas and experiment plans with IdeaAgent, powered by an RL-tuned LLM. Next, ExperimentAgent leverages retrieved prototype code to convert plans into executable code with optionally retrieved candidate models and data from HuggingFace. In the final stage, ExperimentAgent runs experiments, and allows subsequent iterations of debugging and human feedback for a better chance of success with executable outcomes. We evaluate our framework on five machine learning research tasks. Experiment results demonstrate the potential of our framework to facilitate ML research progress and innovation.