Sequential Gradient Descent and Quasi-Newton's Method for Change-Point Analysis

TL;DR

This paper presents a new sequential approach for change-point detection that leverages gradient descent and quasi-Newton methods to significantly speed up computation while maintaining accuracy.

Contribution

The paper introduces SE, a sequential method combined with gradient descent and quasi-Newton techniques, for efficient change-point analysis in large datasets.

Findings

Orders of magnitude faster than PELT

Maintains high estimation accuracy

Effective in generalized linear models and penalized regression

Abstract

One common approach to detecting change-points is minimizing a cost function over possible numbers and locations of change-points. The framework includes several well-established procedures, such as the penalized likelihood and minimum description length. Such an approach requires finding the cost value repeatedly over different segments of the data set, which can be time-consuming when (i) the data sequence is long and (ii) obtaining the cost value involves solving a non-trivial optimization problem. This paper introduces a new sequential method (SE) that can be coupled with gradient descent (SeGD) and quasi-Newton's method (SeN) to find the cost value effectively. The core idea is to update the cost value using the information from previous steps without re-optimizing the objective function. The new method is applied to change-point detection in generalized linear models and penalized regression. Numerical studies show that the new approach can be orders of magnitude faster than the Pruned Exact Linear Time (PELT) method without sacrificing estimation accuracy.