# Explaining an increase in predicted risk for clinical alerts

## TL;DR

This paper develops methods to explain increases in risk predictions over time in dynamic models, with a focus on clinical alerts to help clinicians quickly interpret patient deterioration signals.

## Contribution

It introduces techniques to adapt static attribution methods to temporal models, addressing challenges unique to dynamic risk estimation in clinical settings.

## Key findings

- Expert evaluation shows improved interpretability of alerts.
- Methods effectively attribute risk increases to relevant past inputs.
- Enhanced explanations aid clinical decision-making.

## Abstract

Much work aims to explain a model's prediction on a static input. We consider explanations in a temporal setting where a stateful dynamical model produces a sequence of risk estimates given an input at each time step. When the estimated risk increases, the goal of the explanation is to attribute the increase to a few relevant inputs from the past. While our formal setup and techniques are general, we carry out an in-depth case study in a clinical setting. The goal here is to alert a clinician when a patient's risk of deterioration rises. The clinician then has to decide whether to intervene and adjust the treatment. Given a potentially long sequence of new events since she last saw the patient, a concise explanation helps her to quickly triage the alert. We develop methods to lift static attribution techniques to the dynamical setting, where we identify and address challenges specific to dynamics. We then experimentally assess the utility of different explanations of clinical alerts through expert evaluation.