Discovery and inference beyond linearity by integrating Bayesian regression, tree ensembles and Shapley values

TL;DR

RuleSHAP is a novel framework that combines Bayesian regression, tree rules, and Shapley values to detect nonlinear effects and interactions in healthcare data with reliable uncertainty quantification.

Contribution

It introduces RuleSHAP, a new method integrating Bayesian sparse regression and tree-based rules to enable valid inference of feature effects in complex models.

Findings

Successfully detects nonlinear and interaction effects in simulated data.

Identifies significant effects in epidemiological cohort data.

Provides uncertainty quantification for individual feature effects.

Abstract

Machine Learning (ML) is gaining popularity for hypothesis-free discovery of risk and protective factors in healthcare studies. ML is strong at discovering nonlinearities and interactions, but this power is compromised by a lack of reliable inference. Although Shapley values provide local measures of features' effects, valid uncertainty quantification for these effects is typically lacking, thus precluding statistical inference. We propose RuleSHAP, a framework that addresses this limitation by combining a dedicated Bayesian sparse regression model with a new tree-based rule generator and Shapley value attribution. RuleSHAP provides detection of nonlinear and interaction effects with uncertainty quantification at the individual level. We derive an efficient formula for computing marginal Shapley values within this framework. We demonstrate the validity of our framework on simulated data. Finally, we apply RuleSHAP to data from an epidemiological cohort to detect and infer several effects for high cholesterol and blood pressure, such as nonlinear interaction effects between features like age, sex, ethnicity, BMI and glucose level.