Sobolev Regularized MMD Gradient Flow

TL;DR

This paper introduces Sobolev-regularized MMD gradient flow, which improves convergence guarantees and applicability in both sampling and generative modeling by using a novel regularization approach.

Contribution

It proposes a new Sobolev-regularized MMD gradient flow with provable global convergence that applies to both sampling and generative modeling without relying on isoperimetric assumptions.

Findings

The regularized flow achieves global convergence guarantees.

It is effective in both sampling and generative modeling tasks.

Empirical results demonstrate improved performance across various tasks.

Abstract

We propose Sobolev-regularized Maximum Mean Discrepancy (SrMMD) gradient flow, a regularized variant of maximum mean discrepancy (MMD) gradient flow based on a gradient penalty on the witness function. The proposed regularization mitigates the non-convexity of the MMD objective and yields provable \emph{global} convergence guarantees in MMD in both continuous and discrete time. A more surprising appeal is that our convergence analysis does not rely on isoperimetric assumptions on the target distribution. Instead, it is based on a regularity condition on the difference between kernel mean embeddings. A key highlight of the proposed flow is that it is applicable in both sampling (from an unnormalized target distribution) -- using Stein kernels -- and generative modeling settings, unlike previous works, where a gradient flow is suitable for only generative modeling or sampling but not both. The effectiveness of the proposed flow is empirically verified on a broad range of tasks in both generative modelling and sampling.