# Online estimation of the asymptotic variance for averaged stochastic gradient algorithms

## TL;DR

This paper proves a Central Limit Theorem for stochastic gradient algorithms in Hilbert spaces, introduces a recursive method to estimate their asymptotic variance, and demonstrates its effectiveness through logistic regression and geometric quantile examples.

## Contribution

It establishes a CLT for averaged stochastic gradient estimates in Hilbert spaces and proposes a new recursive algorithm for asymptotic variance estimation.

## Key findings

- Proves asymptotic normality of stochastic gradient estimates.
- Introduces a recursive algorithm for variance estimation.
- Demonstrates the method on logistic regression and geometric quantiles.

## Abstract

Stochastic gradient algorithms are more and more studied since they can deal efficiently and online with large samples in high dimensional spaces. In this paper, we first establish a Central Limit Theorem for these estimates as well as for their averaged version in general Hilbert spaces. Moreover, since having the asymptotic normality of estimates is often unusable without an estimation of the asymptotic variance, we introduce a new recursive algorithm for estimating this last one, and we establish its almost sure rate of convergence as well as its rate of convergence in quadratic mean. Finally, two examples consisting in estimating the parameters of the logistic regression and estimating geometric quantiles are given.