Generalized Kernel Thinning

TL;DR

This paper enhances the kernel thinning algorithm by providing tighter, dimension-free guarantees for various kernels and distributions, and demonstrates improved performance in high-dimensional integration tasks.

Contribution

It introduces four key improvements to kernel thinning, including direct application to RKHS, MMD guarantees for analytic and non-smooth kernels, and a combined approach called KT+.

Findings

Significant reduction in integration error in 100-dimensional problems.

Improved MMD guarantees for analytic kernels like Gaussian and sinc.

Enhanced performance in compressing complex differential equation posteriors.

Abstract

The kernel thinning (KT) algorithm of Dwivedi and Mackey (2021) compresses a probability distribution more effectively than independent sampling by targeting a reproducing kernel Hilbert space (RKHS) and leveraging a less smooth square-root kernel. Here we provide four improvements. First, we show that KT applied directly to the target RKHS yields tighter, dimension-free guarantees for any kernel, any distribution, and any fixed function in the RKHS. Second, we show that, for analytic kernels like Gaussian, inverse multiquadric, and sinc, target KT admits maximum mean discrepancy (MMD) guarantees comparable to or better than those of square-root KT without making explicit use of a square-root kernel. Third, we prove that KT with a fractional power kernel yields better-than-Monte-Carlo MMD guarantees for non-smooth kernels, like Laplace and Mat\'ern, that do not have square-roots. Fourth, we establish that KT applied to a sum of the target and power kernels (a procedure we call KT+) simultaneously inherits the improved MMD guarantees of power KT and the tighter individual function guarantees of target KT. In our experiments with target KT and KT+, we witness significant improvements in integration error even in $100$ dimensions and when compressing challenging differential equation posteriors.