Fast DPP Sampling for Nystr\"om with Application to Kernel Methods

TL;DR

This paper introduces a fast DPP-based landmark selection method for the Nystr"om approximation in kernel methods, providing theoretical error bounds and demonstrating superior empirical performance with linear-time sampling under certain conditions.

Contribution

It develops a linear-time DPP sampling algorithm for landmark selection in Nystr"om, with theoretical guarantees and improved empirical results over existing methods.

Findings

DPP landmarks guarantee bounds on approximation errors

Markov chain DPP sampling can be linear in data size under certain conditions

DPP-based landmark selection outperforms existing approaches in experiments

Abstract

The Nystr\"om method has long been popular for scaling up kernel methods. Its theoretical guarantees and empirical performance rely critically on the quality of the landmarks selected. We study landmark selection for Nystr\"om using Determinantal Point Processes (DPPs), discrete probability models that allow tractable generation of diverse samples. We prove that landmarks selected via DPPs guarantee bounds on approximation errors; subsequently, we analyze implications for kernel ridge regression. Contrary to prior reservations due to cubic complexity of DPPsampling, we show that (under certain conditions) Markov chain DPP sampling requires only linear time in the size of the data. We present several empirical results that support our theoretical analysis, and demonstrate the superior performance of DPP-based landmark selection compared with existing approaches.