Towards Understanding Gradient Dynamics of the Sliced-Wasserstein Distance via Critical Point Analysis

TL;DR

This paper analyzes the critical points of the Sliced Wasserstein Distance, revealing stability properties and convergence behaviors, which are essential for optimizing models in machine learning applications.

Contribution

It provides a rigorous theoretical analysis of the critical points of the SW objective, including stability and convergence, supported by numerical experiments.

Findings

Stable critical points cannot concentrate on segments.

Critical point stability is crucial for optimization algorithms.

Numerical experiments validate theoretical insights.

Abstract

In this paper, we investigate the properties of the Sliced Wasserstein Distance (SW) when employed as an objective functional. The SW metric has gained significant interest in the optimal transport and machine learning literature, due to its ability to capture intricate geometric properties of probability distributions while remaining computationally tractable, making it a valuable tool for various applications, including generative modeling and domain adaptation. Our study aims to provide a rigorous analysis of the critical points arising from the optimization of the SW objective. By computing explicit perturbations, we establish that stable critical points of SW cannot concentrate on segments. This stability analysis is crucial for understanding the behaviour of optimization algorithms for models trained using the SW objective. Furthermore, we investigate the properties of the SW objective, shedding light on the existence and convergence behavior of critical points. We illustrate our theoretical results through numerical experiments.