Breaking Down the Scoring: Interrater Reliability and National Bias in Olympic Breaking

TL;DR

This study examines the reliability and national bias in Olympic Breaking judging, revealing low interrater consistency and significant favoritism towards athletes from judges' own countries, which may impact fairness.

Contribution

It provides the first empirical analysis of judging reliability and national bias in Olympic Breaking, highlighting areas for scoring system improvement.

Findings

Low to moderate interrater reliability compared to other sports.

Significant national bias favoring judges' own countries.

The scoring aggregation reduces individual judge influence.

Abstract

Introduction: The inclusion of Breaking in the 2024 Paris Olympic Games introduced a distinctive competition format in which two athletes compete head-to-head in battle rounds. Unlike other aesthetic sports, where athletes receive independent scores for their performances, Breaking judges assess the relative performance quality between two competitors across five predefined evaluation aspects. This may considerably increase their cognitive load and warrants a thorough examination of judging reliability and potential national bias, the tendency of judges to favor athletes from their own country. Methods: Official scoring data from the 2024 Olympic Breaking events were analyzed. Interrater reliability was assessed using the Intraclass Correlation Coefficient, while national bias was estimated using mixed-effects modelling based on judges' individual scores. Results: The analyses revealed low to moderate interrater reliability in Breaking, notably lower than levels reported in other aesthetic sports. Moreover, substantial national bias was detected, with judges displaying a systematic tendency to favor athletes from their own country. Discussion: Although the observed national bias, despite its notable magnitude, likely had only a limited effect on final competition outcomes, the relatively low interrater reliability highlights potential weaknesses in the current scoring framework that may warrant refinement. Nonetheless, Breaking's unique system of aggregating judges' scores into discrete votes appears to reduce the influence of individual judges on competition outcomes, thereby enhancing the system's robustness against unilateral score manipulation.