Toward Interpretable Evaluation Measures for Time Series Segmentation

TL;DR

This paper introduces two new evaluation measures, WARI and SMS, for time series segmentation that improve interpretability and accuracy over traditional metrics by accounting for error position and type.

Contribution

It proposes novel, interpretable evaluation metrics for time series segmentation, addressing limitations of existing point-based measures.

Findings

WARI accounts for the position of segmentation errors.

SMS identifies and scores four fundamental segmentation error types.

The new measures outperform traditional metrics on synthetic and real data.

Abstract

Time series segmentation is a fundamental task in analyzing temporal data across various domains, from human activity recognition to energy monitoring. While numerous state-of-the-art methods have been developed to tackle this problem, the evaluation of their performance remains critically limited. Existing measures predominantly focus on change point accuracy or rely on point-based measures such as Adjusted Rand Index (ARI), which fail to capture the quality of the detected segments, ignore the nature of errors, and offer limited interpretability. In this paper, we address these shortcomings by introducing two novel evaluation measures: WARI (Weighted Adjusted Rand Index), that accounts for the position of segmentation errors, and SMS (State Matching Score), a fine-grained measure that identifies and scores four fundamental types of segmentation errors while allowing error-specific weighting. We empirically validate WARI and SMS on synthetic and real-world benchmarks, showing that they not only provide a more accurate assessment of segmentation quality but also uncover insights, such as error provenance and type, that are inaccessible with traditional measures.