On ergodicity of the SAGA-LD algorithm

TL;DR

This paper proves that the SAGA-LD algorithm, used for sampling in high-dimensional machine learning, converges to a limiting distribution despite complex dynamics.

Contribution

It introduces a model-specific proof demonstrating the convergence and law of large numbers for SAGA-LD, a challenging sampling algorithm.

Findings

SAGA-LD converges to a limiting distribution.

A law of large numbers holds for SAGA-LD.

Standard Markov chain approaches do not apply due to complex dynamics.

Abstract

The so-called SAGA-LD algorithm is used for efficient sampling from high-dimensional distributions in machine learning. Its intricate dynamics resists standard approaches of Markov chain theory. We prove, using a model-specific method, that SAGA-LD converges to a limiting distribution and a law of large numbers holds.