AstroReview: An LLM-driven Multi-Agent Framework for Telescope Proposal Peer Review and Refinement

TL;DR

AstroReview is an open-source multi-agent framework that automates telescope proposal peer review, improving transparency, consistency, and efficiency without domain-specific fine tuning, and enhances proposal quality through iterative feedback.

Contribution

The paper introduces AstroReview, a novel agent-based system for automating telescope proposal review stages, demonstrating high accuracy and iterative refinement capabilities.

Findings

87% accuracy in identifying accepted proposals

66% increase in acceptance rate after two refinement iterations

Effective automation of proposal review stages without domain-specific tuning

Abstract

Competitive access to modern observatories has intensified as proposal volumes outpace available telescope time, making timely, consistent, and transparent peer review a critical bottleneck for the advancement of astronomy. Automating parts of this process is therefore both scientifically significant and operationally necessary to ensure fair allocation and reproducible decisions at scale. We present AstroReview, an open-source, agent-based framework that automates proposal review in three stages: (i) novelty and scientific merit, (ii) feasibility and expected yield, and (iii) meta-review and reliability verification. Task isolation and explicit reasoning traces curb hallucinations and improve transparency. Without any domain specific fine tuning, AstroReview used in our experiments only for the last stage, correctly identifies genuinely accepted proposals with an accuracy of 87%. The AstroReview in Action module replicates the review and refinement loop; with its integrated Proposal Authoring Agent, the acceptance rate of revised drafts increases by 66% after two iterations, showing that iterative feedback combined with automated meta-review and reliability verification delivers measurable quality gains. Together, these results point to a practical path toward scalable, auditable, and higher throughput proposal review for resource limited facilities.