Minimizing $f$-Divergences by Interpolating Velocity Fields

TL;DR

This paper introduces a method to directly interpolate velocity fields for minimizing $f$-divergences, improving accuracy in distribution approximation tasks like domain adaptation and data imputation.

Contribution

It proposes a novel interpolation-based approach for estimating velocity fields directly, avoiding overfitting issues of previous density ratio methods.

Findings

Interpolated velocity fields are consistent estimators under mild conditions.

The method outperforms previous approaches in domain adaptation tasks.

Effective in missing data imputation scenarios.

Abstract

Many machine learning problems can be seen as approximating a \textit{target} distribution using a \textit{particle} distribution by minimizing their statistical discrepancy. Wasserstein Gradient Flow can move particles along a path that minimizes the $f$-divergence between the target and particle distributions. To move particles, we need to calculate the corresponding velocity fields derived from a density ratio function between these two distributions. Previous works estimated such density ratio functions and then differentiated the estimated ratios. These approaches may suffer from overfitting, leading to a less accurate estimate of the velocity fields. Inspired by non-parametric curve fitting, we directly estimate these velocity fields using interpolation techniques. We prove that our estimators are consistent under mild conditions. We validate their effectiveness using novel applications on domain adaptation and missing data imputation.