Reducing training time by efficient localized kernel regression

TL;DR

This paper explores a partitioning approach to kernel regularized least squares regression, demonstrating that it maintains optimal convergence rates while significantly reducing computational costs through local Nyström subsampling.

Contribution

It introduces a novel partitioning method combined with local Nyström subsampling, enhancing efficiency without sacrificing statistical performance.

Findings

Optimal convergence rates are maintained with slow growth of local sets.

Partitioning combined with local Nyström subsampling halves computational costs.

The approach is effective for large-scale kernel regression problems.

Abstract

We study generalization properties of kernel regularized least squares regression based on a partitioning approach. We show that optimal rates of convergence are preserved if the number of local sets grows sufficiently slowly with the sample size. Moreover, the partitioning approach can be efficiently combined with local Nystr\"om subsampling, improving computational cost twofold.