Metric Entropy and Minimax Risk of Ellipsoids with an Application to Pinsker's Theorem

TL;DR

This paper introduces type-$ au$ integrals to analyze the size of $ ext{ell}^2$ ellipsoids, linking metric entropy and minimax risk, and applies these results to improve classical theorems like Pinsker's in non-parametric estimation.

Contribution

It systematically relates type-$ au$ integrals to metric entropy and minimax risk, sharpening classical results and extending Pinsker's theorem to broader settings.

Findings

Asymptotic equivalence of metric entropy and type-1 integral.

Minimax risk characterized by type-2 and type-3 integrals.

Improved bounds for Sobolev ellipsoid metric entropy.

Abstract

We study how large an $\ell^2$ ellipsoid is by introducing type-$\tau$ integrals that capture the average decay of its semi-axes. These integrals turn out to be closely related to standard complexity measures: we show that the metric entropy of the ellipsoid is asymptotically equivalent to the type-1 integral, and that the minimax risk in non-parametric estimation is asymptotically determined by the type-2 and type-3 integrals. This allows us to retrieve and sharpen classical results about metric entropy and minimax risk of ellipsoids through a systematic analysis of the type-$\tau$ integrals, and yields an explicit formula linking the two. As an application, we improve on the best-known characterization of the metric entropy of the Sobolev ellipsoid, and extend Pinsker's Sobolev theorem in two ways: (i) to any bounded open domain in arbitrary finite dimension, and (ii) by providing the second-order term in the asymptotic expansion of the minimax risk.