Local optimisation of Nystr\"om samples through stochastic gradient descent

TL;DR

This paper introduces a stochastic gradient descent method to optimize Nystr"om samples based on a radial squared-kernel discrepancy criterion, leading to improved kernel matrix approximations.

Contribution

It proposes a novel local optimization approach for Nystr"om samples using stochastic gradient descent on a relaxed SKD criterion, enhancing approximation accuracy.

Findings

Optimized Nystr"om samples improve kernel approximation accuracy.

The stochastic gradient descent method is efficient for sample optimization.

Numerical experiments confirm the effectiveness of the approach.

Abstract

We study a relaxed version of the column-sampling problem for the Nystr\"om approximation of kernel matrices, where approximations are defined from multisets of landmark points in the ambient space; such multisets are referred to as Nystr\"om samples. We consider an unweighted variation of the radial squared-kernel discrepancy (SKD) criterion as a surrogate for the classical criteria used to assess the Nystr\"om approximation accuracy; in this setting, we discuss how Nystr\"om samples can be efficiently optimised through stochastic gradient descent. We perform numerical experiments which demonstrate that the local minimisation of the radial SKD yields Nystr\"om samples with improved Nystr\"om approximation accuracy.