Near-Optimal Coresets of Kernel Density Estimates

TL;DR

This paper develops near-optimal coresets for kernel density estimates in high-dimensional spaces, providing polynomial-time constructions with size bounds that improve upon previous results and depend on the dimension and accuracy.

Contribution

It introduces a polynomial-time method for constructing near-optimal coresets for positive definite kernels, with size bounds that are close to the theoretical lower bounds and depend on dimension and error.

Findings

Coreset size is $O(rac{ oot{d}}{ argeteps}\sqrt{\log 1/ argeteps})$

Lower bounds show size must be at least $ ilde{ argeteps}^{-2}$ or depend on $ oot{d}$

The results apply to a wide class of kernels used in machine learning

Abstract

We construct near-optimal coresets for kernel density estimates for points in $\mathbb{R}^d$ when the kernel is positive definite. Specifically we show a polynomial time construction for a coreset of size $O(\sqrt{d}/\varepsilon\cdot \sqrt{\log 1/\varepsilon} )$, and we show a near-matching lower bound of size $\Omega(\min\{\sqrt{d}/\varepsilon, 1/\varepsilon^2\})$. When $d\geq 1/\varepsilon^2$, it is known that the size of coreset can be $O(1/\varepsilon^2)$. The upper bound is a polynomial-in-$(1/\varepsilon)$ improvement when $d \in [3,1/\varepsilon^2)$ and the lower bound is the first known lower bound to depend on $d$ for this problem. Moreover, the upper bound restriction that the kernel is positive definite is significant in that it applies to a wide-variety of kernels, specifically those most important for machine learning. This includes kernels for information distances and the sinc kernel which can be negative.