Competing with AI Scientists: Agent-Driven Approach to Astrophysics Research

TL;DR

This paper introduces an agent-driven system for constructing scientific data analysis pipelines, demonstrating its effectiveness in a cosmology challenge and showing potential to compete with expert solutions.

Contribution

It presents a multi-agent framework that collaborates to generate, execute, and refine research pipelines, integrating human intervention to achieve top results in a cosmological inference challenge.

Findings

Semi-autonomous agents achieved first place in a cosmology challenge.

The workflow combines neural networks, likelihood calibration, and regularization techniques.

Agent-driven workflows can rapidly explore and build inference pipelines.

Abstract

We present an agent-driven approach to the construction of parameter inference pipelines for scientific data analysis. Our method leverages a multi-agent system, Cmbagent (the analysis system of the AI scientist Denario), in which specialized agents collaborate to generate research ideas, write and execute code, evaluate results, and iteratively refine the overall pipeline. As a case study, we apply this approach to the FAIR Universe Weak Lensing Uncertainty Challenge, a competition under time constraints focused on robust cosmological parameter inference with realistic observational uncertainties. While the fully autonomous exploration initially did not reach expert-level performance, the integration of human intervention enabled our agent-driven workflow to achieve a first-place result in the challenge. This demonstrates that semi-autonomous agentic systems can compete with, and in some cases surpass, expert solutions. We describe our workflow in detail, including both the autonomous and semi-autonomous exploration by Cmbagent. Our final inference pipeline utilizes parameter-efficient convolutional neural networks, likelihood calibration over a known parameter grid, and multiple regularization techniques. Our results suggest that agent-driven research workflows can provide a scalable framework to rapidly explore and construct pipelines for inference problems.