A vector-contraction inequality for Rademacher complexities

TL;DR

This paper extends the contraction inequality for Rademacher averages to vector-valued functions, allowing for broader applications in machine learning tasks like multi-category learning and clustering.

Contribution

It generalizes the contraction inequality to vector-valued functions and replaces Rademacher variables with sub-gaussian variables in bounds.

Findings

Extended contraction inequality to vector-valued functions

Replaced Rademacher variables with sub-gaussian variables in bounds

Applied results to multi-category learning, K-means, and learning-to-learn

Abstract

The contraction inequality for Rademacher averages is extended to Lipschitz functions with vector-valued domains, and it is also shown that in the bounding expression the Rademacher variables can be replaced by arbitrary iid symmetric and sub-gaussian variables. Example applications are given for multi-category learning, K-means clustering and learning-to-learn.