Vector-valued self-normalized concentration inequalities beyond sub-Gaussianity

TL;DR

This paper develops new concentration inequalities for vector-valued self-normalized processes with light tails beyond sub-Gaussian assumptions, with applications in online linear regression and bandits.

Contribution

It introduces novel concentration bounds for vector-valued processes outside the sub-Gaussian framework, extending the theory of self-normalized inequalities.

Findings

Provides concentration bounds for processes with Bennett or Bernstein tails.

Demonstrates applications in online linear regression and kernelized linear bandits.

Extends the understanding of self-normalized concentration beyond scalar and sub-Gaussian cases.

Abstract

The study of self-normalized processes plays a crucial role in a wide range of applications, from sequential decision-making to econometrics. While the behavior of self-normalized concentration has been widely investigated for scalar-valued processes, vector-valued processes remain comparatively underexplored, especially outside of the sub-Gaussian framework. In this contribution, we provide concentration bounds for self-normalized processes with light tails beyond sub-Gaussianity (such as Bennett or Bernstein bounds). We illustrate the relevance of our results in the context of online linear regression, with applications in (kernelized) linear bandits.