Berry-Esseen theorems for the asymptotic normality of incomplete U-statistics with Bernoulli sampling

TL;DR

This paper establishes Berry-Esseen bounds for the asymptotic normality of incomplete U-statistics with Bernoulli sampling, providing theoretical guarantees under minimal assumptions for their use in machine learning uncertainty quantification.

Contribution

It introduces new Berry-Esseen bounds for incomplete U-statistics with Bernoulli sampling, covering different regimes and utilizing novel proof techniques.

Findings

Berry-Esseen bounds with natural rates established

Normal approximation accuracy characterized under minimal assumptions

Applicable to uncertainty quantification in machine learning

Abstract

There has been a resurgence of interest in incomplete U-statistics that only sum over a subset of kernel evaluations, due to their computational efficiency and asymptotic normality which can be leveraged to quantify the uncertainty of ensemble predictions in machine learning. In this paper, we study the weak convergences to normality of one such construction, the incomplete U-statistic with Bernoulli sampling, under three different regimes on the relative sizes of the raw sample and the computational budget. Under minimalistic moment assumptions, we establish accompanying Berry-Esseen bounds with the natural rates that characterize the accuracy of these normal approximations. The key ingredients in our proofs include a variable censoring technique and a methodology for establishing Berry-Esseen bounds for the so-called Studentized nonlinear statistics recently formalized in the Stein's method literature, as well as an exponential lower tail bound for non-negative kernel U-statistics.