Inferring Change Points in Regression via Sample Weighting

TL;DR

This paper introduces Weighted ERM, a novel method for detecting change points in high-dimensional generalized linear models using sample weighting, with theoretical analysis and practical implementation.

Contribution

It proposes a new sample-weighted empirical risk minimization approach for change point detection, with asymptotic performance characterization and an open-source implementation.

Findings

Weighted ERM accurately detects change points in simulations.

The method outperforms existing approaches in real data experiments.

Sample weights with weak priors improve change point estimation.

Abstract

We study the problem of identifying change points in high-dimensional generalized linear models, and propose an approach based on sample-weighted empirical risk minimization. Our method, Weighted ERM, encodes priors on the change points via weights assigned to each sample, to obtain weighted versions of standard estimators such as M-estimators and maximum-likelihood estimators. Under mild assumptions on the data, we obtain a precise asymptotic characterization of the performance of our method for general Gaussian designs, in the high-dimensional limit where the number of samples and covariate dimension grow proportionally. We show how this characterization can be used to efficiently construct a posterior distribution over change points. Numerical experiments on both simulated and real data illustrate the efficacy of Weighted ERM compared to existing approaches, demonstrating that sample weights constructed with weakly informative priors can yield accurate change point estimators. Our method is implemented as an open-source package, weightederm, available in Python and R.