Detecting AI-Generated Content in Academic Peer Reviews

TL;DR

This paper investigates the rise of AI-generated content in academic peer reviews over time, revealing a significant increase in detection of such content from 2022 to 2025 at major conferences and journals.

Contribution

It introduces a detection model trained on historical reviews to quantify the emergence and growth of AI-generated peer review content over recent years.

Findings

AI-generated reviews increased to 20% at ICLR in 2025

AI-generated reviews reached 12% at Nature Communications in 2025

Most growth occurred between late 2024 and early 2025

Abstract

The growing availability of large language models (LLMs) has raised questions about their role in academic peer review. This study examines the temporal emergence of AI-generated content in peer reviews by applying a detection model trained on historical reviews to later review cycles at International Conference on Learning Representations (ICLR) and Nature Communications (NC). We observe minimal detection of AI-generated content before 2022, followed by a substantial increase through 2025, with approximately 20% of ICLR reviews and 12% of Nature Communications reviews classified as AI-generated in 2025. The most pronounced growth of AI-generated reviews in NC occurs between the third and fourth quarter of 2024. Together, these findings provide suggestive evidence of a rapidly increasing presence of AI-assisted content in peer review and highlight the need for further study of its implications for scholarly evaluation.