Near-Balanced Incomplete Block Designs with An Application to Poster Competitions

TL;DR

This paper introduces two near-balanced incomplete block designs tailored for poster judging in competitions, addressing challenges of judge assignment and unknown judge numbers, and demonstrates their superior accuracy and efficiency over random assignments.

Contribution

The paper proposes two novel near-balanced incomplete block designs for poster judging that ensure connectedness and treatment balance, improving estimation accuracy and efficiency.

Findings

Both designs outperform random assignment in accuracy.

Designs improve the chance of identifying top posters.

Simulation confirms efficiency gains.

Abstract

Judging scholarly posters creates a challenge to assign the judges efficiently. If there are many posters and few reviews per judge, the commonly used Balanced Incomplete Block Design is not a feasible option. An additional challenge is an unknown number of judges before the event. We propose two connected near-balanced incomplete block designs that both satisfy the requirements of our setting: one that generates a connected assignment and balances the treatments and another one that further balances pairs of treatments. We describe both fixed and random effects models to estimate the population marginal means of the poster scores and rationalize the use of the random effects model. We evaluate the estimation accuracy and efficiency, especially the winning chance of the truly best posters, of the two designs in comparison with a random assignment via simulation studies. The two proposed designs both demonstrate accuracy and efficiency gains over the random assignment.