Tensor Decompositions for Count Data that Leverage Stochastic and Deterministic Optimization

TL;DR

This paper introduces two novel algorithms for fitting low-rank Poisson tensor decompositions, combining stochastic and deterministic optimization techniques to improve convergence to the maximum likelihood estimator.

Contribution

It proposes hybrid and restart-based algorithms that enhance convergence and reduce computational cost in Poisson CPD tensor decomposition.

Findings

Our methods outperform existing algorithms in converging to the Poisson CPD MLE.

Hybrid GCP-CPAPR improves convergence probability over individual methods.

Restarted CPAPR with SVDrop reduces computational costs with effective heuristics.

Abstract

There is growing interest to extend low-rank matrix decompositions to multi-way arrays, or tensors. One fundamental low-rank tensor decomposition is the canonical polyadic decomposition (CPD). The challenge of fitting a low-rank, nonnegative CPD model to Poisson-distributed count data is of particular interest. Several popular algorithms use local search methods to approximate the maximum likelihood estimator (MLE) of the Poisson CPD model. This work presents two new algorithms that extend state-of-the-art local methods for Poisson CPD. Hybrid GCP-CPAPR combines Generalized Canonical Decomposition (GCP) with stochastic optimization and CP Alternating Poisson Regression (CPAPR), a deterministic algorithm, to increase the probability of converging to the MLE over either method used alone. Restarted CPAPR with SVDrop uses a heuristic based on the singular values of the CPD model unfoldings to identify convergence toward optimizers that are not the MLE and restarts within the feasible domain of the optimization problem, thus reducing overall computational cost when using a multi-start strategy. We provide empirical evidence that indicates our approaches outperform existing methods with respect to converging to the Poisson CPD MLE.