Open Source Planning & Control System with Language Agents for Autonomous Scientific Discovery

TL;DR

This paper introduces cmbagent, a multi-agent system utilizing language models to automate complex scientific research tasks without human intervention, demonstrated on cosmology data analysis with superior results.

Contribution

The paper presents a novel multi-agent framework that orchestrates LLMs for autonomous scientific discovery, including task specialization and local code execution, advancing automation in research.

Findings

Successfully applied to cosmology parameter measurement

Outperforms state-of-the-art LLMs on benchmarks

Demonstrates autonomous research workflow execution

Abstract

We present a multi-agent system for automation of scientific research tasks, cmbagent (https://github.com/CMBAgents/cmbagent). The system is formed by about 30 Large Language Model (LLM) agents and implements a Planning & Control strategy to orchestrate the agentic workflow, with no human-in-the-loop at any point. Each agent specializes in a different task (performing retrieval on scientific papers and codebases, writing code, interpreting results, critiquing the output of other agents) and the system is able to execute code locally. We successfully apply cmbagent to carry out a PhD level cosmology task (the measurement of cosmological parameters using supernova data) and evaluate its performance on two benchmark sets, finding superior performance over state-of-the-art LLMs. The source code is available on GitHub, demonstration videos are also available, and the system is deployed on HuggingFace and will be available on the cloud.