On Optimal Multiple Changepoint Algorithms for Large Data

TL;DR

This paper introduces two new algorithms, FPOP and SNIP, that efficiently detect multiple changepoints in large datasets, combining the accuracy of dynamic programming with improved computational speed and robustness.

Contribution

The paper extends pruning methods and presents FPOP and SNIP algorithms, offering faster and more robust changepoint detection in long time-series data.

Findings

FPOP is significantly faster than existing dynamic programming methods.

FPOP's efficiency remains stable regardless of the number of changepoints.

FPOP's performance is competitive with Binary Segmentation in detecting Copy Number Variations.

Abstract

There is an increasing need for algorithms that can accurately detect changepoints in long time-series, or equivalent, data. Many common approaches to detecting changepoints, for example based on penalised likelihood or minimum description length, can be formulated in terms of minimising a cost over segmentations. Dynamic programming methods exist to solve this minimisation problem exactly, but these tend to scale at least quadratically in the length of the time-series. Algorithms, such as Binary Segmentation, exist that have a computational cost that is close to linear in the length of the time-series, but these are not guaranteed to find the optimal segmentation. Recently pruning ideas have been suggested that can speed up the dynamic programming algorithms, whilst still being guaranteed to find true minimum of the cost function. Here we extend these pruning methods, and introduce two new algorithms for segmenting data, FPOP and SNIP. Empirical results show that FPOP is substantially faster than existing dynamic programming methods, and unlike the existing methods its computational efficiency is robust to the number of changepoints in the data. We evaluate the method at detecting Copy Number Variations and observe that FPOP has a computational cost that is competitive with that of Binary Segmentation.