Finite sample deviation and variance bounds for first order autoregressive processes

TL;DR

This paper derives finite-sample deviation and variance bounds for the least squares estimator in first order autoregressive processes, applicable to both stable and unstable cases, validated through simulations.

Contribution

It introduces new non-asymptotic bounds on estimator deviation and variance for AR(1) processes using decoupling theory, covering both stable and unstable scenarios.

Findings

Bounds accurately predict estimator behavior in simulations

Results apply to both stable and unstable AR(1) processes

Bounds are valid for sample sizes ≥ 7

Abstract

In this paper, we study finite-sample properties of the least squares estimator in first order autoregressive processes. By leveraging a result from decoupling theory, we derive upper bounds on the probability that the estimate deviates by at least a positive $\varepsilon$ from its true value. Our results consider both stable and unstable processes. Afterwards, we obtain problem-dependent non-asymptotic bounds on the variance of this estimator, valid for sample sizes greater than or equal to seven. Via simulations we analyze the conservatism of our bounds, and show that they reliably capture the true behavior of the quantities of interest.