Atomic Gradient Flows: Gradient Flows on Sparse Representations

TL;DR

This paper introduces Atomic Gradient Flows (AGFs), a generalization of Particle Gradient Flows, for convex optimization in Banach spaces, enabling efficient sparse representations and extending applications to infinite-dimensional problems.

Contribution

It develops a new framework for gradient flows on sparse representations in Banach spaces, using extremal points and Wasserstein space lifting, with theoretical guarantees and practical applications.

Findings

AGFs are well-defined and consistent with original problems via $ extGamma$-convergence.

The lifted AGF evolution is a metric gradient flow in Wasserstein space.

Applicable to infinite-dimensional problems like functions of bounded variation.

Abstract

One of the most popular approaches for solving total variation-regularized optimization problems in the space of measures are Particle Gradient Flows (PGFs). These restrict the problem to linear combinations of Dirac deltas and then perform a Euclidean gradient flow in the weights and positions, significantly reducing the computational cost while still decreasing the energy. In this work, we generalize PGFs to convex optimization problems in arbitrary Banach spaces, which we call Atomic Gradient Flows (AGFs). To this end, the crucial ingredient turns out to be the right notion of particles, chosen here as the extremal points of the unit ball of the regularizer. This choice is motivated by the Krein-Milman theorem, which ensures that minimizers can be approximated by linear combinations of extremal points. We investigate metric gradient flows of the optimization problem when restricted to such sparse representations, for which we define a suitable discretized functional that we show to be to be consistent with the original problem via the means of $\Gamma$-convergence. We prove that the resulting evolution of the latter is well-defined using a minimizing movement scheme, and we establish conditions ensuring $\lambda$-convexity and uniqueness of the flow. Then, using Choquet's theorem, we lift the problem into the Wasserstein space on weights and extremal points, and consider Wasserstein gradient flows in this lifted setting. Our main result is that the lifting of the AGF evolution is again a metric gradient flow in the Wasserstein space, verifying the consistency of the approach with respect to a Wasserstein-type dynamic. Finally, we illustrate the applicability of AGFs to several relevant infinite-dimensional problems, including optimization of functions of bounded variation and curves of measures regularized by Optimal Transport-type penalties.