Evaluating the methodological suitability of partial dependence plots and Shapley additive explanations for population-level interpretation of machine learning models in total joint arthroplasty

TL;DR

This study compares two methods for interpreting machine learning models in joint replacement surgery, finding that partial dependence plots are better for creating population-level clinical guidelines.

Contribution

The study evaluates PDPs and SHAP for population-level interpretation in TJA risk prediction, highlighting PDPs' suitability for clinical guideline development.

Findings

PDPs showed 97.8% agreement with SHAP in directional trends, validating their comparison.

PDPs revealed actionable thresholds for preoperative hematocrit, operative time, and feature interactions like age × ASA classification.

Interaction effects accounted for nearly half of the model's influence, emphasizing the importance of complementary relationships.

Abstract

Total joint arthroplasty (TJA) complications necessitate the development of accurate risk prediction models; however, interpretability in machine learning remains a challenge. While Shapley Additive Explanations (SHAP) offers insights at the individual level, partial dependence plots (PDPs) may provide a better understanding at the population level for developing clinical guidelines. This study compared PDPs and SHAP in explaining machine learning-based 30-day complication risk prediction following TJA.

We conducted a retrospective cohort study using the American College of Surgeons National Surgical Quality Improvement Program (NSQIP) database (2019–2023), including 517,826 primary TJA cases. Binary classification models (Random Forest, Gradient Boosting) predicted composite 30-day complications based on 20 clinical predictors. A comprehensive interpretability analysis employed directional concordance validation between PDP and SHAP, permutation importance thresholding (5% relative influence), followed by one- and two-dimensional partial dependence analyses with explicit interaction modeling.

The cohort comprised 517,826 primary TJA procedures with a complication rate of 6.67%. The baseline Random Forest model achieved test AUC = 0.678. Directional concordance analysis demonstrated 97.8% weighted agreement between PDP trends and SHAP attributions, validating methodological comparison. Threshold analysis identified seven significant features, with interaction effects accounting for 49.9% of total model influence (71.9% among top features). PDPs showed actionable dose–response relationships, including critical thresholds for preoperative hematocrit (< 38%), operative time (> 120 min), and complementary interactions, such as age × ASA classification (19.1% importance), operative time × ASA classification (10.1%), and hematocrit × diabetes (6.4%). Comparative patient analysis demonstrated that while SHAP quantified individual contributions, only PDPs provided population thresholds directly translatable to institutional protocols.

PDPs appear more methodologically appropriate than SHAP for population-level clinical guideline development, offering actionable dose–response relationships and population risk thresholds that SHAP’s individualized attribution framework cannot provide. The dominance of interaction effects among the most influential predictors validates that PDPs accurately capture complementary relationships while presenting them in a format directly applicable to evidence-based perioperative protocols and institutional quality improvement initiatives.

Video Abstract

Video Abstract

The online version contains supplementary material available at 10.1186/s42836-025-00360-9.

Figures