Sinkhorn Based Associative Memory Retrieval Using Spherical Hellinger Kantorovich Dynamics

TL;DR

This paper introduces a novel dense associative memory model for probability measures, utilizing Sinkhorn divergence and spherical Hellinger Kantorovich dynamics to achieve robust pattern retrieval with high capacity.

Contribution

It develops a new retrieval algorithm based on SHK gradient flow for weighted point clouds, with theoretical guarantees and high-dimensional capacity analysis.

Findings

Proves basin invariance and geometric convergence under certain conditions.

Demonstrates exponential capacity in high dimensions.

Shows robust recovery in synthetic experiments.

Abstract

We propose a dense associative memory for empirical measures (weighted point clouds). Stored patterns and queries are finitely supported probability measures, and retrieval is defined by minimizing a Hopfield-style log-sum-exp energy built from the debiased Sinkhorn divergence. We derive retrieval dynamics as a spherical Hellinger Kantorovich (SHK) gradient flow, which updates both support locations and weights. Discretizing the flow yields a deterministic algorithm that uses Sinkhorn potentials to compute barycentric transport steps and a multiplicative simplex reweighting. Under local separation and PL-type conditions we prove basin invariance, geometric convergence to a local minimizer, and a bound showing the minimizer remains close to the corresponding stored pattern. Under a random pattern model, we further show that these Sinkhorn basins are disjoint with high probability, implying exponential capacity in the ambient dimension. Experiments on synthetic Gaussian point-cloud memories demonstrate robust recovery from perturbed queries versus a Euclidean Hopfield-type baseline.