On the Optimality of Misspecified Spectral Algorithms

TL;DR

This paper investigates the optimality of spectral algorithms in misspecified settings, demonstrating they are minimax optimal across a broad range of smoothness levels in RKHSs, including cases previously unresolved.

Contribution

The authors prove spectral algorithms are minimax optimal for all smoothness levels s in (0,1) under certain conditions, extending prior results.

Findings

Spectral algorithms are minimax optimal for s in (α₀ - 1/β, 1).

Identified classes of RKHSs where α₀ equals 1/β, ensuring optimality for all s in (0,1).

Extended the understanding of spectral algorithms' optimality beyond previous smoothness constraints.

Abstract

In the misspecified spectral algorithms problem, researchers usually assume the underground true function $f_{\rho}^{*} \in [\mathcal{H}]^{s}$, a less-smooth interpolation space of a reproducing kernel Hilbert space (RKHS) $\mathcal{H}$ for some $s\in (0,1)$. The existing minimax optimal results require $\|f_{\rho}^{*}\|_{L^{\infty}}<\infty$ which implicitly requires $s > \alpha_{0}$ where $\alpha_{0}\in (0,1)$ is the embedding index, a constant depending on $\mathcal{H}$. Whether the spectral algorithms are optimal for all $s\in (0,1)$ is an outstanding problem lasting for years. In this paper, we show that spectral algorithms are minimax optimal for any $\alpha_{0}-\frac{1}{\beta} < s < 1$, where $\beta$ is the eigenvalue decay rate of $\mathcal{H}$. We also give several classes of RKHSs whose embedding index satisfies $ \alpha_0 = \frac{1}{\beta} $. Thus, the spectral algorithms are minimax optimal for all $s\in (0,1)$ on these RKHSs.