The Counterfactual Combine: A Causal Framework for Player Evaluation

TL;DR

This paper introduces a causal inference framework for sports player evaluation, adapting methods from healthcare profiling to compare player success rates under hypothetical reassignments, providing more interpretable performance metrics.

Contribution

It develops a flexible causal evaluation framework using stochastic interventions and doubly robust estimators, integrating machine learning for complex relationship modeling in sports analytics.

Findings

Case studies on NFL kickers and MLB batters demonstrate the framework's insights.

Different causal estimands offer varied interpretations of player performance.

The approach improves accuracy and interpretability of player evaluation metrics.

Abstract

Evaluating sports players based on their performance shares core challenges with evaluating healthcare providers based on patient outcomes. Drawing on recent advances in healthcare provider profiling, we cast sports player evaluation within a rigorous causal inference framework and define a flexible class of causal player evaluation estimands. Using stochastic interventions, we compare player success rates on repeated tasks (such as field goal attempts or plate appearance) to counterfactual success rates had those same attempts been randomly reassigned to players according to prespecified reference distributions. This setup encompasses direct and indirect standardization parameters familiar from healthcare provider profiling, and we additionally propose a "performance above random replacement" estimand designed for interpretability in sports settings. We develop doubly robust estimators for these evaluation metrics based on modern semiparametric statistical methods, with a focus on Targeted Minimum Loss-based Estimation, and incorporate machine learning methods to capture complex relationships driving player performance. We illustrate our framework in detailed case studies of field goal kickers in the National Football League and batters in Major League Baseball, highlighting how different causal estimands yield distinct interpretations and insights about player performance.