Robust variance estimators in application to segmentation of measurement data distorted by impulsive and non-Gaussian noise

TL;DR

This paper introduces a robust, variance-independent method for detecting regime change points in measurement data affected by impulsive and non-Gaussian noise, improving accuracy in various real-world systems.

Contribution

It formulates a new offline robust segmentation methodology based on classical structural break detection, effective without requiring data variance, applicable to diverse systems.

Findings

Error in change point estimation reduced by 20 times in challenging cases

Method effective on simulated and real-world datasets

Outperforms classical approaches in impulsive noise environments

Abstract

The paper algorithmizes the problem of regime change point identification for data measured in a system exhibiting impulsive behaviors. This is a fundamental challenge for annotation of measurement data relevant, e.g., for designing data-driven autonomous systems. The contribution consists in the formulation of an offline robust methodology based on the classical approach for structural break detection. The problem of data segmentation is considered in the context of scale change, which physically can be translated into the occurrence of a critical event that reorganizes the system structure. The main advantage of our approach is that it does not require the existence of a variance of the data distribution. The efficiency has been evaluated for simulated data from two distributions and for real-world datasets measured in financial, mechanical, and medical systems. Simulation studies show that in the most challenging case, the error in estimating regime change is 20 times smaller for robust approach compared to the classical one.