Semi-Implicit Functional Gradient Flow for Efficient Sampling

TL;DR

This paper introduces Semi-Implicit Functional Gradient Flow (SIFG), a particle-based sampling method that uses Gaussian-perturbed particles and score matching for improved exploration and convergence in variational inference.

Contribution

The paper proposes a novel semi-implicit functional gradient flow method that leverages Gaussian perturbations and neural network score matching, providing theoretical guarantees and adaptive noise selection.

Findings

Demonstrates improved sampling efficiency on simulated datasets.

Achieves better exploration compared to deterministic methods.

Shows strong convergence guarantees due to higher-order smoothness.

Abstract

Particle-based variational inference methods (ParVIs) use nonparametric variational families represented by particles to approximate the target distribution according to the kernelized Wasserstein gradient flow for the Kullback-Leibler (KL) divergence. Although functional gradient flows have been introduced to expand the kernel space for better flexibility, the deterministic updating mechanism may limit exploration and require expensive repetitive runs for new samples. In this paper, we propose Semi-Implicit Functional Gradient flow (SIFG), a functional gradient ParVI method that uses perturbed particles with Gaussian noise as the approximation family. We show that the corresponding functional gradient flow, which can be estimated via denoising score matching with neural networks, exhibits strong theoretical convergence guarantees due to a higher-order smoothness brought to the approximation family via Gaussian perturbation. In addition, we present an adaptive version of our method that automatically selects the appropriate noise magnitude during sampling, striking a good balance between exploration efficiency and approximation accuracy. Extensive experiments on both simulated and real-world datasets demonstrate the effectiveness and efficiency of the proposed framework.