Parametric kernel low-rank approximations using tensor train decomposition

TL;DR

This paper introduces a new efficient method for low-rank approximation of parametric kernel matrices using tensor train decomposition and Chebyshev approximation, enabling fast computations in scientific and data science applications.

Contribution

The authors develop a novel approach combining Chebyshev approximation with tensor train decomposition for parametric kernel matrices, achieving linear complexity and significant speedups.

Findings

Achieves up to 200x speedup in online computations.

Maintains comparable accuracy to existing methods.

Effective for various kernels like Matérn kernel.

Abstract

Computing low-rank approximations of kernel matrices is an important problem with many applications in scientific computing and data science. We propose methods to efficiently approximate and store low-rank approximations to kernel matrices that depend on certain hyperparameters. The main idea behind our method is to use multivariate Chebyshev function approximation along with the tensor train decomposition of the coefficient tensor. The computations are in two stages: an offline stage, which dominates the computational cost and is parameter-independent, and an online stage, which is inexpensive and instantiated for specific hyperparameters. A variation of this method addresses the case that the kernel matrix is symmetric and positive semi-definite. The resulting algorithms have linear complexity in terms of the sizes of the kernel matrices. We investigate the efficiency and accuracy of our method on parametric kernel matrices induced by various kernels, such as the Mat\'ern kernel, through various numerical experiments. Our methods have speedups up to $200\times$ in the online time compared to other methods with similar complexity and comparable accuracy.