Delta-XAI: A Unified Framework for Explaining Prediction Changes in Online Time Series Monitoring

TL;DR

Delta-XAI introduces a unified framework and evaluation suite for explaining prediction changes in online time series models, emphasizing temporal dependencies and proposing a novel method called SWING.

Contribution

It adapts 14 XAI methods for online time series, introduces SWING for temporal explanations, and provides a comprehensive evaluation suite for online settings.

Findings

Classical gradient-based methods like Integrated Gradients outperform recent approaches in temporal analysis.

SWING effectively captures temporal dependencies and reduces out-of-distribution effects.

Experiments show SWING's robustness across diverse online time series scenarios.

Abstract

Explaining online time series monitoring models is crucial across sensitive domains such as healthcare and finance, where temporal and contextual prediction dynamics underpin critical decisions. While recent XAI methods have improved the explainability of time series models, they mostly analyze each time step independently, overlooking temporal dependencies. This results in further challenges: explaining prediction changes is non-trivial, methods fail to leverage online dynamics, and evaluation remains difficult. To address these challenges, we propose Delta-XAI, which adapts 14 existing XAI methods through a wrapper function and introduces a principled evaluation suite for the online setting, assessing diverse aspects, such as faithfulness, sufficiency, and coherence. Experiments reveal that classical gradient-based methods, such as Integrated Gradients (IG), can outperform recent approaches when adapted for temporal analysis. Building on this, we propose Shifted Window Integrated Gradients (SWING), which incorporates past observations in the integration path to systematically capture temporal dependencies and mitigate out-of-distribution effects. Extensive experiments consistently demonstrate the effectiveness of SWING across diverse settings with respect to diverse metrics. Our code is publicly available at https://github.com/AITRICS/Delta-XAI.