How to monitor and mitigate stair-casing in l1 trend filtering

TL;DR

This paper investigates the stair-casing problem in $ ext{l}_1$ trend filtering for time-series trend detection, analyzing its causes, and proposing modifications to improve false change point detection accuracy.

Contribution

It identifies the stair-case effect in $ ext{l}_1$ trend filtering and proposes monitoring and algorithmic modifications to mitigate false change point detection.

Findings

$ ext{l}_1$ trend filtering suffers from stair-case effects.

Dual variable interpretation as integrated random walk.

Proposed modifications reduce false change point detection.

Abstract

In this paper we study the estimation of changing trends in time-series using $\ell_1$ trend filtering. This method generalizes 1D Total Variation (TV) denoising for detection of step changes in means to detecting changes in trends, and it relies on a convex optimization problem for which there are very efficient numerical algorithms. It is known that TV denoising suffers from the so-called stair-case effect, which leads to detecting false change points. The objective of this paper is to show that $\ell_1$ trend filtering also suffers from a certain stair-case problem. The analysis is based on an interpretation of the dual variables of the optimization problem in the method as integrated random walk. We discuss consistency conditions for $\ell_1$ trend filtering, how to monitor their fulfillment, and how to modify the algorithm to avoid the stair-case false detection problem.