Proximal Interacting Particle Langevin Algorithms

TL;DR

This paper introduces proximal interacting particle Langevin algorithms (PIPLA) for efficient inference in non-differentiable latent variable models, providing theoretical bounds and demonstrating effectiveness in various applications.

Contribution

The paper develops a new class of algorithms combining proximal MCMC and interacting particle Langevin methods for non-differentiable models, with theoretical guarantees and practical demonstrations.

Findings

Nonasymptotic bounds for parameter estimates in strongly log-concave settings

Effective in sparse Bayesian logistic regression and neural network training

Demonstrates utility in image deblurring and matrix completion

Abstract

We introduce a class of algorithms, termed proximal interacting particle Langevin algorithms (PIPLA), for inference and learning in latent variable models whose joint probability density is non-differentiable. Leveraging proximal Markov chain Monte Carlo techniques and interacting particle Langevin algorithms, we propose three algorithms tailored to the problem of estimating parameters in a non-differentiable statistical model. We prove nonasymptotic bounds for the parameter estimates produced by the different algorithms in the strongly log-concave setting and provide comprehensive numerical experiments on various models to demonstrate the effectiveness of the proposed methods. In particular, we demonstrate the utility of our family of algorithms for sparse Bayesian logistic regression, training of sparse Bayesian neural networks or neural networks with non-differentiable activation functions, image deblurring, and sparse matrix completion. Our theory and experiments together show that PIPLA family can be the de facto choice for parameter estimation problems in non-differentiable latent variable models.