Fast Summation of Radial Kernels via QMC Slicing
Johannes Hertrich, Tim Jahn, Michael Quellmalz
TL;DR
This paper introduces a QMC-based slicing method for fast summation of radial kernels, leveraging random projections and Fourier techniques, resulting in improved efficiency over existing methods.
Contribution
It proposes a novel QMC slicing approach with theoretical error bounds for efficient large-scale kernel sum computation.
Findings
QMC slicing outperforms existing kernel approximation methods
Theoretical bounds validate the slicing error control
Numerical experiments confirm significant performance improvements
Abstract
The fast computation of large kernel sums is a challenging task, which arises as a subproblem in any kernel method. We approach the problem by slicing, which relies on random projections to one-dimensional subspaces and fast Fourier summation. We prove bounds for the slicing error and propose a quasi-Monte Carlo (QMC) approach for selecting the projections based on spherical quadrature rules. Numerical examples demonstrate that our QMC-slicing approach significantly outperforms existing methods like (QMC-)random Fourier features, orthogonal Fourier features or non-QMC slicing on standard test datasets.
Peer Reviews
Decision·ICLR 2025 Poster
Although slicing has been thoroughly explored in reducing the complexity in time of optimal transport, slicing the computation of kernels is a novel topic, very rencelty introduced. - The novel error bound differs from Hertrich (2024, SIAM Journal on Mathematics of Data Science and arxiv) with a characterization of the vairance of the slicing estimator, which confirms the rate in O(1/sqrt(P)). - The most interesting and promising part from my point of view is related to the exploration of quadra
- About the motivation:The number of data is usually considered as the most emblematic issue with kernels in Machine Learning and existing approximation schemes aim at reducing the compute time involved by operations with the Gram matrix as well as the complexity in memory. The authors motivate their work by the case when the dimension of data is large. Contrary to optimal transport problems that are defined in a variational way, computing sums of kernels at the era of distributed computing is
1.The QMC slicing method introduces an approach to fast kernel summation. 2.The methodology is well-documented, with explanations of error bounds and smoothness results. 3.QMC slicing has the potential to improve efficiency in kernel-based methods.
1.The adaptability of the theoretical error bounds to various types of data and kernels remains unclear. 2.While effective, the QMC slicing method lacks clear differentiation from existing fast summation methods, such as Random Fourier Features (RFF). 3.Experiments are focused on limited, synthetic datasets, raising concerns about the method's generalizability. 4.Some parts of the theory section are overly condensed, especially around error bounds and smoothness assumptions.
- The theoretical analysis in the paper seemingly technically sound, though I couldn't follow the proofs.
- Overall, the paper is difficult to follow those who are not familiar with the topic. Readability can be improved. - Empirical evaluation is only on purely kernel sum calculations. Showing benefit on actual computation of some learning model would have been convincing.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsImage Processing Techniques and Applications · Underwater Acoustics Research · Face and Expression Recognition