xaitimesynth: A Python Package for Evaluating Attribution Methods for Time Series with Synthetic Ground Truth

TL;DR

xaitimesynth is a Python package that simplifies the evaluation of time series attribution methods using synthetic data with known ground truth, supporting flexible dataset creation and standard metrics.

Contribution

It introduces a reusable, flexible Python tool for generating synthetic time series with known feature locations for attribution method evaluation.

Findings

Provides a standardized framework for synthetic data generation.

Includes multiple localization metrics for attribution evaluation.

Supports both univariate and multivariate time series.

Abstract

Evaluating time series attribution methods is difficult because real-world datasets rarely provide ground truth for which time points drive a prediction. A common workaround is to generate synthetic data where class-discriminating features are placed at known locations, but each study currently reimplements this from scratch. We introduce xaitimesynth, a Python package that provides reusable infrastructure for this evaluation approach. The package generates synthetic time series following an additive model where each sample is a sum of background signal and a localized, class-discriminating feature, with the feature window automatically tracked as a ground truth mask. A fluent data generation API and YAML configuration format allow flexible and reproducible dataset definitions for both univariate and multivariate time series. The package also provides standard localization metrics, including AUC-PR, AUC-ROC, Relevance Mass Accuracy, and Relevance Rank Accuracy. xaitimesynth is open source and available at https://github.com/gregorbaer/xaitimesynth.