Trend Filtering -- I. A Modern Statistical Tool for Time-Domain Astronomy and Astronomical Spectroscopy

TL;DR

This paper introduces trend filtering, a modern statistical method for denoising one-dimensional signals with varying smoothness, significantly improving analysis in time-domain astronomy and spectroscopy.

Contribution

The paper presents trend filtering as a scalable, nonparametric tool that outperforms traditional methods for denoising spatially heterogeneous signals in astronomical data.

Findings

Superior performance over kernel smoothers and splines

Efficient computation for large datasets (n > 10^7)

Effective in diverse astronomical applications

Abstract

The problem of denoising a one-dimensional signal possessing varying degrees of smoothness is ubiquitous in time-domain astronomy and astronomical spectroscopy. For example, in the time domain, an astronomical object may exhibit a smoothly varying intensity that is occasionally interrupted by abrupt dips or spikes. Likewise, in the spectroscopic setting, a noiseless spectrum typically contains intervals of relative smoothness mixed with localized higher frequency components such as emission peaks and absorption lines. In this work, we present trend filtering, a modern nonparametric statistical tool that yields significant improvements in this broad problem space of denoising $spatially$ $heterogeneous$ signals. When the underlying signal is spatially heterogeneous, trend filtering is superior to any statistical estimator that is a linear combination of the observed data---including kernel smoothers, LOESS, smoothing splines, Gaussian process regression, and many other popular methods. Furthermore, the trend filtering estimate can be computed with practical and scalable efficiency via a specialized convex optimization algorithm, e.g. handling sample sizes of $n\gtrsim10^7$ within a few minutes. In a companion paper, we explicitly demonstrate the broad utility of trend filtering to observational astronomy by carrying out a diverse set of spectroscopic and time-domain analyses.