Performance analysis of greedy algorithms for minimising a Maximum Mean Discrepancy

TL;DR

This paper compares the performance of greedy algorithms for minimizing Maximum Mean Discrepancy in probability measure quantization, highlighting trade-offs between accuracy and computational efficiency.

Contribution

It provides a theoretical analysis of the convergence rates of kernel herding, greedy MMD minimization, and SBQ, with practical insights on their computational costs.

Findings

SBQ achieves a $1/n$ error decay rate.

Kernel herding and greedy MMD also achieve $1/n$ decay with proper step-size.

SBQ has a slightly better error bound but higher computational cost.

Abstract

We analyse the performance of several iterative algorithms for the quantisation of a probability measure $\mu$, based on the minimisation of a Maximum Mean Discrepancy (MMD). Our analysis includes kernel herding, greedy MMD minimisation and Sequential Bayesian Quadrature (SBQ). We show that the finite-sample-size approximation error, measured by the MMD, decreases as $1/n$ for SBQ and also for kernel herding and greedy MMD minimisation when using a suitable step-size sequence. The upper bound on the approximation error is slightly better for SBQ, but the other methods are significantly faster, with a computational cost that increases only linearly with the number of points selected. This is illustrated by two numerical examples, with the target measure $\mu$ being uniform (a space-filling design application) and with $\mu$ a Gaussian mixture. They suggest that the bounds derived in the paper are overly pessimistic, in particular for SBQ. The sources of this pessimism are identified but seem difficult to counter.