Clinical outcome prediction under hypothetical interventions -- a representation learning framework for counterfactual reasoning

TL;DR

This paper presents a novel representation learning framework that enables clinical outcome prediction with integrated counterfactual reasoning, facilitating personalized care and decision-making in healthcare.

Contribution

It introduces a partial concept bottleneck framework that incorporates counterfactual explanations into predictive models without sacrificing accuracy.

Findings

Comparable prediction accuracy to traditional models

Potential to improve personalized clinical decision-making

Enables investigation of hypothetical intervention effects

Abstract

Most machine learning (ML) models are developed for prediction only; offering no option for causal interpretation of their predictions or parameters/properties. This can hamper the health systems' ability to employ ML models in clinical decision-making processes, where the need and desire for predicting outcomes under hypothetical investigations (i.e., counterfactual reasoning/explanation) is high. In this research, we introduce a new representation learning framework (i.e., partial concept bottleneck), which considers the provision of counterfactual explanations as an embedded property of the risk model. Despite architectural changes necessary for jointly optimising for prediction accuracy and counterfactual reasoning, the accuracy of our approach is comparable to prediction-only models. Our results suggest that our proposed framework has the potential to help researchers and clinicians improve personalised care (e.g., by investigating the hypothetical differential effects of interventions)