The Three Little Pigs and the Big Bad Wolf: Case Studies of Peer Review

TL;DR

This paper analyzes three peer review cases involving theoretical construction proposals for houses, highlighting the review process, reviewer biases, and the potential influence of reviewer self-interest on rejection decisions.

Contribution

It provides a detailed case study of peer review dynamics using a whimsical analogy, revealing possible biases and motivations behind review outcomes.

Findings

Review decisions varied based on manuscript complexity and style.

Reviewer biases may influence rejection, independent of manuscript quality.

Self-interest and personal motives can impact peer review judgments.

Abstract

I present for your appraisal three independent cases of the manuscript referee process conducted by a venerable peer-reviewed scientific journal. Each case involves a little pig, who submitted for consideration a theoretical plan for a house to be constructed presently, in a faraway land. An anonymous big bad wolf was assigned by the journal to assess the merit of these manuscripts. The pigs proposed three distinct construction frameworks, which varied in physical and mathematical sophistication. The first little pig submitted a model of straw, based on the numerical method of toe-counting. His design included odd features, such as spilled millet and cloven-hoofprints on the window sill -- possibly a ploy to distract the wolf from the manuscript's facile mathematical foundation. The second little pig used a more advanced approach, employing Newton's classical laws of motion, to propose a house of sticks. This pig included in her manuscript copious citations by a specific wolf, possibly aiming to ensure acceptance by flattering the wolf whom she anticipated would be the referee. The third little pig described an ostentatious house of bricks based on an elaborate dynamical systems and stability analysis, possibly scheming to dazzle and impress. The big bad wolf did not appear moved by any of the pigs' tactics. His recommendations were, for straw: the minor revision of water-proofing; for sticks: the major revision of fire-proofing, given concerns surrounding climate change; for bricks: unequivocal rejection, accompanied by multiple derogatory comments regarding "high-and-mighty theorists." I describe each case in detail, and suggest that the wolf's reports might be driven as much by self interest as the manuscripts themselves -- namely, that at the time the wolf wrote the reviews, he was rather hungry. Finally, I examine morals learned, if any.