Meta-Analytics: Tools for Understanding the Statistical Properties of Sports Metrics

TL;DR

This paper introduces 'meta-metrics' to evaluate sports metrics based on stability, discrimination, and independence, helping analysts identify the most reliable and unique metrics for decision-making.

Contribution

The paper develops practical methods to assess and compare sports metrics using three key criteria, aiding in the construction and selection of effective performance measures.

Findings

Identified the most reliable NBA and NHL metrics

Provided insights on constructing independent and stable metrics

Demonstrated methods are easy to implement and broadly applicable

Abstract

In sports, there is a constant effort to improve metrics which assess player ability, but there has been almost no effort to quantify and compare existing metrics. Any individual making a management, coaching, or gambling decision is quickly overwhelmed with hundreds of statistics. We address this problem by proposing a set of "meta-metrics" which can be used to identify the metrics that provide the most unique, reliable, and useful information for decision-makers. Specifically, we develop methods to evalute metrics based on three criteria: 1) stability: does the metric measure the same thing over time 2) discrimination: does the metric differentiate between players and 3) independence: does the metric provide new information? Our methods are easy to implement and widely applicable so they should be of interest to the broader sports community. We demonstrate our methods in analyses of both NBA and NHL metrics. Our results indicate the most reliable metrics and highlight how they should be used by sports analysts. The meta-metrics also provide useful insights about how to best construct new metrics which provide independent and reliable information about athletes.