Dimension-free uniform concentration bound for logistic regression

TL;DR

This paper introduces a new dimension-free uniform concentration bound for constrained logistic regression, improving conditions for uniform law of large numbers using PAC-Bayes and Rademacher complexity techniques.

Contribution

It presents a novel, milder sufficient condition for uniform convergence in logistic regression, leveraging PAC-Bayes and second-order expansion methods.

Findings

Provides a dimension-free concentration bound for logistic regression

Yields milder conditions for uniform law of large numbers

Uses PAC-Bayes approach with second-order expansion

Abstract

We provide a novel dimension-free uniform concentration bound for the empirical risk function of constrained logistic regression. Our bound yields a milder sufficient condition for a uniform law of large numbers than conditions derived by the Rademacher complexity argument and McDiarmid's inequality. The derivation is based on the PAC-Bayes approach with second-order expansion and Rademacher-complexity-based bounds for the residual term of the expansion.