A Bennett Inequality for the Missing Mass

TL;DR

This paper introduces new concentration inequalities for the missing mass, providing tighter deviation bounds especially for small deviations, which are crucial in learning theory applications.

Contribution

It presents novel, distribution-free deviation bounds for the missing mass with improved accuracy over previous results, particularly for small deviations.

Findings

Derived tighter bounds for missing mass deviations

Improved results over prior inequalities for small deviations

Applicable to learning theory scenarios

Abstract

Novel concentration inequalities are obtained for the missing mass, i.e. the total probability mass of the outcomes not observed in the sample. We derive distribution-free deviation bounds with sublinear exponents in deviation size for missing mass and improve the results of Berend and Kontorovich (2013) and Yari Saeed Khanloo and Haffari (2015) for small deviations which is the most important case in learning theory.