Analysis of regularized Nystr\"om subsampling for regression functions of low smoothness

TL;DR

This paper investigates a Nyström subsampling method for large-scale kernel regression when the target function is not in the associated RKHS, revealing that effective learning rates can be achieved with simple regularization even in misspecified cases.

Contribution

It provides new theoretical insights into Nyström subsampling for misspecified kernel learning, showing that a single regularization parameter suffices across various source conditions.

Findings

Learning rates are achievable with one regularization parameter across many source conditions.

Nyström subsampling can be implemented with subquadratic computational cost.

The approach maintains guaranteed learning rates even in misspecified scenarios.

Abstract

This paper studies a Nystr\"om type subsampling approach to large kernel learning methods in the misspecified case, where the target function is not assumed to belong to the reproducing kernel Hilbert space generated by the underlying kernel. This case is less understood, in spite of its practical importance. To model such a case, the smoothness of target functions is described in terms of general source conditions. It is surprising that almost for the whole range of the source conditions, describing the misspecified case, the corresponding learning rate bounds can be achieved with just one value of the regularization parameter. This observation allows a formulation of mild conditions under which the plain Nystr\"om subsampling can be realized with subquadratic cost maintaining the guaranteed learning rates.