Advancing statistical decision-making in sports science

TL;DR

This paper critically evaluates the magnitude-based decisions (MBD) method in sports science, providing a theoretical foundation, visualisation tools, and demonstrating its potential to improve decision-making and interpretation in sports research.

Contribution

It generalises funnel plots for effect size ranges, relates MBD to decision procedures, and offers a visualisation tool to support better statistical decision-making in sports science.

Findings

MBD can be grounded in a solid theoretical framework.

Visualisation tools enhance understanding of effect sizes.

MBD encourages more nuanced and planned statistical testing.

Abstract

The magnitude-based decisions (MBD) procedure was developed within sports science as an alternative to null hypothesis significance tests. It aimed to emphasise effect sizes and discourage dichotomous decision-making. The use of MBD was banned by some sports science journals following claims it lacks a theoretical foundation and leads to high Type I error rates. To address these claims, we first generalise contour-enhanced funnel plots to allow for ranges of meaningful effect sizes, then relate regions defined in these plots to the decisions made by MBD. We then mathematically show how MBD fits within a class of multiple decision procedures. We have implemented this theoretically sound version of MBD as a visualisation tool that supports generalised funnel plots. The use of MBD could encourage researchers to plan test directionalities, test levels and error definitions, and the visualisation tool may help stakeholders engage with the design of analyses and the interpretation of trial findings.