Towards Empirical Process Theory for Vector-Valued Functions: Metric Entropy of Smooth Function Classes

TL;DR

This paper develops empirical process theory for vector-valued functions, providing entropy bounds for smooth classes in Hilbert spaces, and explores implications for statistical learning and complexity measures.

Contribution

It introduces entropy bounds for smooth vector-valued function classes and extends empirical process tools to Hilbert space-valued functions.

Findings

Entropy bounds enable uniform law of large numbers for vector-valued functions

Results facilitate asymptotic analysis in Hilbert space-valued statistical models

Discussion on extending Rademacher complexities to vector-valued functions

Abstract

This paper provides some first steps in developing empirical process theory for functions taking values in a vector space. Our main results provide bounds on the entropy of classes of smooth functions taking values in a Hilbert space, by leveraging theory from differential calculus of vector-valued functions and fractal dimension theory of metric spaces. We demonstrate how these entropy bounds can be used to show the uniform law of large numbers and asymptotic equicontinuity of the function classes, and also apply it to statistical learning theory in which the output space is a Hilbert space. We conclude with a discussion on the extension of Rademacher complexities to vector-valued function classes.