AI-Assisted Scientific Assessment: A Case Study on Climate Change

TL;DR

This study demonstrates how AI can assist climate scientists in synthesizing complex research, significantly accelerating workflow while requiring expert oversight for scientific rigor.

Contribution

The paper introduces a Gemini-based AI environment tailored for collaborative scientific assessment, tested in climate science with positive results.

Findings

AI accelerated the synthesis of 79 papers in 46 person-hours.

Most AI-generated content was retained in the final report.

Expert oversight was essential for scientific rigor and quality.

Abstract

The emerging paradigm of AI co-scientists focuses on tasks characterized by repeatable verification, where agents explore search spaces in 'guess and check' loops. This paradigm does not extend to problems where repeated evaluation is impossible and ground truth is established by the consensus synthesis of theory and existing evidence. We evaluate a Gemini-based AI environment designed to support collaborative scientific assessment, integrated into a standard scientific workflow. In collaboration with a diverse group of 13 scientists working in the field of climate science, we tested the system on a complex topic: the stability of the Atlantic Meridional Overturning Circulation (AMOC). Our results show that AI can accelerate the scientific workflow. The group produced a comprehensive synthesis of 79 papers through 104 revision cycles in just over 46 person-hours. AI contribution was significant: most AI-generated content was retained in the report. AI also helped maintain logical consistency and presentation quality. However, expert additions were crucial to ensure its acceptability: less than half of the report was produced by AI. Furthermore, substantial oversight was required to expand and elevate the content to rigorous scientific standards.