Tail bounds for empirically standardized sums

TL;DR

This paper investigates tail bounds for sums standardized using empirical methods, showing that proper Studentization can restore exponential tail decay, with applications to various statistical tests and heteroscedastic data analysis.

Contribution

It introduces new tail bounds for empirically standardized sums, including a novel scan statistic for heteroscedastic data with log-concave distributions.

Findings

Studentized sums can have sub-Gaussian tail bounds

A new scan statistic for heteroscedastic data is proposed

Tail bounds are established for different standardization methods

Abstract

Exponential tail bounds for sums play an important role in statistics, but the example of the $t$-statistic shows that the exponential tail decay may be lost when population parameters need to be estimated from the data. However, it turns out that if Studentizing is accompanied by estimating the location parameter in a suitable way, then the $t$-statistic regains the exponential tail behavior. Motivated by this example, the paper analyzes other ways of empirically standardizing sums and establishes tail bounds that are sub-Gaussian or even closer to normal for the following settings: Standardization with Studentized contrasts for normal observations, standardization with the log likelihood ratio statistic for observations from an exponential family, and standardization via self-normalization for observations from a symmetric distribution with unknown center of symmetry. The latter standardization gives rise to a novel scan statistic for heteroscedastic data whose asymptotic power is analyzed in the case where the observations have a log-concave distribution.