The Perturbed Prox-Preconditioned SPIDER algorithm for EM-based large scale learning

TL;DR

The paper introduces 3P-SPIDER, a novel stochastic EM algorithm that efficiently handles large-scale learning with intractable E-steps, non-smooth regularization, and convex constraints, outperforming existing methods.

Contribution

The paper proposes 3P-SPIDER, a new stochastic EM algorithm that extends SPIDER-EM to address intractable E-steps, regularization, and constraints in large-scale learning.

Findings

3P-SPIDER outperforms existing incremental EM algorithms in experiments.

The algorithm effectively manages intractable E-steps and non-smooth regularization.

Design parameters significantly influence the algorithm's performance.

Abstract

Incremental Expectation Maximization (EM) algorithms were introduced to design EM for the large scale learning framework by avoiding the full data set to be processed at each iteration. Nevertheless, these algorithms all assume that the conditional expectations of the sufficient statistics are explicit. In this paper, we propose a novel algorithm named Perturbed Prox-Preconditioned SPIDER (3P-SPIDER), which builds on the Stochastic Path Integral Differential EstimatoR EM (SPIDER-EM) algorithm. The 3P-SPIDER algorithm addresses many intractabilities of the E-step of EM; it also deals with non-smooth regularization and convex constraint set. Numerical experiments show that 3P-SPIDER outperforms other incremental EM methods and discuss the role of some design parameters.