Online Infinite-Dimensional Regression: Learning Linear Operators

TL;DR

This paper investigates the online learnability of linear operators between infinite-dimensional Hilbert spaces, identifying conditions under which learning is possible or impossible, and highlighting differences between uniform convergence and learnability.

Contribution

It establishes the online learnability of certain classes of linear operators with bounded p-Schatten norm and demonstrates impossibility results for others, revealing fundamental learnability boundaries.

Findings

Linear operators with bounded p-Schatten norm are online learnable.

Operators bounded in operator norm are not online learnable.

Separation between uniform convergence and online learnability is demonstrated.

Abstract

We consider the problem of learning linear operators under squared loss between two infinite-dimensional Hilbert spaces in the online setting. We show that the class of linear operators with uniformly bounded $p$-Schatten norm is online learnable for any $p \in [1, \infty)$. On the other hand, we prove an impossibility result by showing that the class of uniformly bounded linear operators with respect to the operator norm is \textit{not} online learnable. Moreover, we show a separation between sequential uniform convergence and online learnability by identifying a class of bounded linear operators that is online learnable but uniform convergence does not hold. Finally, we prove that the impossibility result and the separation between uniform convergence and learnability also hold in the batch setting.