Learning with Differentially Private (Sliced) Wasserstein Gradients

TL;DR

This paper presents a new framework for differentially private optimization using Wasserstein gradients, enabling privacy-preserving deep learning with theoretical guarantees and practical effectiveness.

Contribution

It introduces a novel sensitivity analysis of Wasserstein gradients for privacy guarantees and develops a deep learning method with gradient clipping for private optimal transport-based objectives.

Findings

Effective privacy-utility trade-off demonstrated in experiments

Framework extends privacy accounting to Wasserstein-based objectives

Enables large-scale private training with strong theoretical guarantees

Abstract

In this work, we introduce a novel framework for privately optimizing objectives that rely on Wasserstein distances between data-dependent empirical measures. Our main theoretical contribution is, based on an explicit formulation of the Wasserstein gradient in a fully discrete setting, a control on the sensitivity of this gradient to individual data points, allowing strong privacy guarantees at minimal utility cost. Building on these insights, we develop a deep learning approach that incorporates gradient and activations clipping, originally designed for DP training of problems with a finite-sum structure. We further demonstrate that privacy accounting methods extend to Wasserstein-based objectives, facilitating large-scale private training. Empirical results confirm that our framework effectively balances accuracy and privacy, offering a theoretically sound solution for privacy-preserving machine learning tasks relying on optimal transport distances such as Wasserstein distance or sliced-Wasserstein distance.