Accurate Evaluation of Quickest Changepoint Detectors via Non-parametric Survival Analysis

TL;DR

This paper introduces non-parametric estimators for ARL and ADD in quickest changepoint detection, leveraging survival analysis to improve robustness and interpretability with limited or irregular data sequences.

Contribution

It proposes KM-ARL and KM-ADD estimators that address practical challenges in applying QCD metrics to real-world datasets with irregular lengths.

Findings

Estimators are asymptotically unbiased without extrapolation.

Experiments show improved robustness and interpretability.

Code is available at the provided GitHub URL.

Abstract

We propose non-parametric estimators for the average run length (ARL) and average detection delay (ADD) in quickest changepoint detection (QCD) under finite and irregular sequence lengths. Although ARL and ADD are widely used as optimality criteria in theoretical and simulation studies, their application to real-world datasets is hindered by limited and irregular sequence lengths. To address this issue, we propose non-parametric estimators for the ARL and ADD, termed KM-ARL and KM-ADD, by drawing an analogy between QCD and survival analysis to model detection probabilities under sequence truncation. We derive estimation bias bounds and prove that they are asymptotically unbiased unless extrapolation is required. Experiments on simulated and real-world datasets demonstrate their practical utility, enhancing robustness against limited and irregular sequence lengths, improving interpretability, and facilitating empirical, intuitive model selection. Our Python code is provided at https://github.com/TaikiMiyagawa/Kaplan-Meier-Average-Run-Length, offering ready-to-use implementations for practitioners.