Recovery guarantees for polynomial approximation from dependent data with outliers

TL;DR

This paper establishes theoretical guarantees for polynomial approximation from dependent, noisy data by framing it as a sparse linear regression problem and proving null space properties for the sampling matrix.

Contribution

It provides the first recovery guarantees for polynomial approximation from dependent data using $ ext{l}_1$-optimization, extending results to various dependent data types.

Findings

Sampling matrix satisfies null space property under dependence conditions.

Guarantees apply to exponentially strongly $ ext{α}$-mixing, $ ext{C}$-mixing, and ergodic Markov data.

Numerical simulations verify theoretical recovery results.

Abstract

Learning non-linear systems from noisy, limited, and/or dependent data is an important task across various scientific fields including statistics, engineering, computer science, mathematics, and many more. In general, this learning task is ill-posed; however, additional information about the data's structure or on the behavior of the unknown function can make the task well-posed. In this work, we study the problem of learning nonlinear functions from corrupted and dependent data. The learning problem is recast as a sparse robust linear regression problem where we incorporate both the unknown coefficients and the corruptions in a basis pursuit framework. The main contribution of our paper is to provide a reconstruction guarantee for the associated $\ell_1$-optimization problem where the sampling matrix is formed from dependent data. Specifically, we prove that the sampling matrix satisfies the null space property and the stable null space property, provided that the data is compact and satisfies a suitable concentration inequality. We show that our recovery results are applicable to various types of dependent data such as exponentially strongly $\alpha$-mixing data, geometrically $\mathcal{C}$-mixing data, and uniformly ergodic Markov chain. Our theoretical results are verified via several numerical simulations.