# Markov Decision Processes with Dynamic Transition Probabilities: An Analysis of Shooting Strategies in Basketball

## TL;DR

This paper models basketball plays as non-stationary Markov decision processes with shot clock-dependent transition probabilities, using Bayesian hierarchical models to analyze and simulate game strategies and their efficiency.

## Contribution

It introduces a novel approach combining lineup-specific MDPs into team-average models with a new transition weighting scheme, enabling realistic simulation of basketball strategies.

## Key findings

- Simulated seasons reveal impacts of different decision policies on team efficiency.
- Bayesian models effectively borrow strength across players and time.
- Team-average MDPs facilitate computationally feasible analysis of non-stationary game dynamics.

## Abstract

In this paper we model basketball plays as episodes from team-specific non-stationary Markov decision processes (MDPs) with shot clock dependent transition probabilities. Bayesian hierarchical models are employed in the modeling and parametrization of the transition probabilities to borrow strength across players and through time. To enable computational feasibility, we combine lineup-specific MDPs into team-average MDPs using a novel transition weighting scheme. Specifically, we derive the dynamics of the team-average process such that the expected transition count for an arbitrary state-pair is equal to the weighted sum of the expected counts of the separate lineup-specific MDPs. We then utilize these non-stationary MDPs in the creation of a basketball play simulator with uncertainty propagated via posterior samples of the model components. After calibration, we simulate seasons both on-policy and under altered policies and explore the net changes in efficiency and production under the alternate policies. Additionally, we discuss the game-theoretic ramifications of testing alternative decision policies.