Causal Analysis of Author Demographics in Academic Peer Review

TL;DR

This study uses causal inference to reveal significant biases against minority race, gender, and geographic location in academic peer review, highlighting the need for fairness interventions to promote equity.

Contribution

It introduces a causal analysis framework to quantify the independent effects of author demographics on paper acceptance, revealing biases in peer review processes.

Findings

Authors from minority racial groups face a -0.42 ranking disadvantage.

Female authors experience a -0.25 ranking effect.

Authors from the Global South face a -0.57 ranking disadvantage.

Abstract

Academic meritocracy is jeopardized by systematic imbalances; for example, whereas Black and Hispanic individuals constitute over 30% of the U.S. population, they represent fewer than 10% of tenured academics in science and engineering. Peer review serves as a crucial gatekeeper in this process, however it encounters ongoing issues over biases that may hinder scientific advancement. The issue is now exacerbated by the growing influence of artificial intelligence (AI) in academic assessment. This paper transcends correlation to quantitatively assess the independent impacts of author demographics, including race, gender, and country of affiliation, on paper acceptance rankings. We utilize a causal inference methodology on a dataset of 530 papers, simulating the academic selection process by employing the prestige of the publication venue as a surrogate for review rank. Our research indicates statistically substantial causal disadvantages for authors from minority racial groups (average treatment effects [ATE]: -0.42 points in ranking), female authors (ATE: -0.25), and those associated with institutions in the Global South (ATE: -0.57). The exhibited biases emphasize the pressing necessity for fairness interventions in both conventional and AI-based review processes, indicating that such measures are essential for establishing a more equitable and credible scientific environment.