ClimAgent: LLM as Agents for Autonomous Open-ended Climate Science Analysis

TL;DR

ClimAgent is an autonomous framework leveraging LLMs for comprehensive climate science analysis, surpassing simple question-answering to enable end-to-end modeling and discovery across diverse climate tasks.

Contribution

The paper introduces ClimAgent, a versatile LLM-based system with a unified tool-use environment and reasoning protocols, along with ClimaBench, a benchmark for real-world climate research tasks.

Findings

ClimAgent outperforms state-of-the-art baselines by 40.21% on ClimaBench.

ClimAgent demonstrates significant improvements in solution rigor and practicality.

The framework enables complex climate research beyond simple question-answering.

Abstract

Climate research is pivotal for mitigating global environmental crises, yet the accelerating volume of multi-scale datasets and the complexity of analytical tools have created significant bottlenecks, constraining scientific discovery to fragmented and labor-intensive workflows. While the emergence Large Language Models (LLMs) offers a transformative paradigm to scale scientific expertise, existing explorations remain largely confined to simple Question-Answering (Q&A) tasks. These approaches often oversimplify real-world challenges, neglecting the intricate physical constraints and the data-driven nature required in professional climate science.To bridge this gap, we introduce ClimAgent, a general-purpose autonomous framework designed to execute a wide spectrum of research tasks across diverse climate sub-fields. By integrating a unified tool-use environment with rigorous reasoning protocols, ClimAgent transcends simple retrieval to perform end-to-end modeling and analysis. To foster systematic evaluation, we propose ClimaBench, the first comprehensive benchmark for real-world climate discovery. It encompasses challenging problems spanning 5 distinct task categories derived from professional scenarios between 2000 and 2025. Experiments on ClimaBench demonstrate that ClimAgent significantly outperforms state-of-the-art baselines, achieving a 40.21% improvement over original LLM solutions in solution rigorousness and practicality. Our code are available at https://github.com/usail-hkust/ClimAgent.