Gender and Discipline Shape Length, Content and Tone of Grant Peer Review Reports

TL;DR

This study analyzes how gender and discipline influence the length, content, and tone of grant peer review reports, revealing significant differences that impact fairness and evaluation consistency.

Contribution

It provides a large-scale, data-driven analysis of how gender and discipline shape review report characteristics using machine learning methods.

Findings

Reviews in SSH are longer and more critical.

Female reviewers write longer, more criteria-aligned, and more positive reviews.

Female applicants receive slightly more positive reviews.

Abstract

Peer review by experts is central to the evaluation of grant proposals, but little is known about how gender and disciplinary differences shape the content and tone of grant peer review reports. We analyzed 39,280 review reports submitted to the Swiss National Science Foundation between 2016 and 2023, covering 11,385 proposals for project funding across 21 disciplines from the Social Sciences and Humanities (SSH), Life Sciences (LS), and Mathematics, Informatics, Natural Sciences, and Technology (MINT). Using supervised machine learning, we classified over 1.3 million sentences by evaluation criteria and sentiment. Reviews in SSH were significantly longer and more critical, with less focus on the applicant's track record, while those in MINT were more concise and positive, with a higher focus on the track record, as compared to those in LS. Compared to male reviewers, female reviewers write longer reviews that more closely align with the evaluation criteria and express more positive sentiments. Female applicants tend to receive reviews with slightly more positive sentiment than male applicants. Gender and disciplinary culture influence how grant proposals are reviewed - shaping the tone, length, and focus of peer review reports. These differences have important implications for fairness and consistency in research funding.