Finite impulse response models: A non-asymptotic analysis of the least squares estimator

TL;DR

This paper provides non-asymptotic, near-optimal bounds for least-squares estimation in finite impulse response models with dependent, sub-Gaussian covariates, extending known results to dependent data.

Contribution

It introduces new concentration inequalities for dependent covariates, generalizing existing bounds for independent data in finite impulse response models.

Findings

Derived near-optimal estimation bounds for dependent covariates.

Extended known bounds to dependent, sub-Gaussian design matrices.

Provided concentration inequalities based on covariance operator singular values.

Abstract

We consider a finite impulse response system with centered independent sub-Gaussian design covariates and noise components that are not necessarily identically distributed. We derive non-asymptotic near-optimal estimation and prediction bounds for the least-squares estimator of the parameters. Our results are based on two concentration inequalities on the norm of sums of dependent covariate vectors and on the singular values of their covariance operator that are of independent value on their own and where the dependence arises from the time shift structure of the time series. These results generalize the known bounds for the independent case.