Importance sampling-based gradient method for dimension reduction in Poisson log-normal model

TL;DR

This paper introduces a new importance sampling-based gradient method for dimension reduction in Poisson log-normal models, providing theoretical convergence guarantees and improved scalability over variational inference.

Contribution

It proposes a projected stochastic gradient approach with importance sampling for direct maximum likelihood estimation in high-dimensional Poisson models, with proven convergence.

Findings

Convergence rate of $O(T^{-1/2} + N^{-1})$ established.

Method scales efficiently with data size.

Numerical results show improved performance over variational methods.

Abstract

High-dimensional count data poses significant challenges for statistical analysis, necessitating effective methods that also preserve explainability. We focus on a low rank constrained variant of the Poisson log-normal model, which relates the observed data to a latent low-dimensional multivariate Gaussian variable via a Poisson distribution. Variational inference methods have become a golden standard solution to infer such a model. While computationally efficient, they usually lack theoretical statistical properties with respect to the model. To address this issue we propose a projected stochastic gradient scheme that directly maximizes the log-likelihood. We prove the convergence of the proposed method when using importance sampling for estimating the gradient. Specifically, we obtain a rate of convergence of $O(T^{-1/2} + N^{-1})$ with $T$ the number of iterations and $N$ the number of Monte Carlo draws. The latter follows from a novel descent lemma for non convex $L$-smooth objective functions, and random biased gradient estimate. We also demonstrate numerically the efficiency of our solution compared to its variational competitor. Our method not only scales with respect to the number of observed samples but also provides access to the desirable properties of the maximum likelihood estimator.