On Tensors, Sparsity, and Nonnegative Factorizations

TL;DR

This paper introduces a new tensor factorization method for sparse count data based on a Poisson model, with algorithms and theory ensuring convergence and scalability for large datasets.

Contribution

It proposes the CP-APR algorithm for Poisson tensor factorization, generalizing Lee-Seung updates and providing convergence guarantees under mild conditions.

Findings

CP-APR effectively models sparse count data.

The algorithm scales to large tensors.

Results on real and simulated data demonstrate its utility.

Abstract

Tensors have found application in a variety of fields, ranging from chemometrics to signal processing and beyond. In this paper, we consider the problem of multilinear modeling of sparse count data. Our goal is to develop a descriptive tensor factorization model of such data, along with appropriate algorithms and theory. To do so, we propose that the random variation is best described via a Poisson distribution, which better describes the zeros observed in the data as compared to the typical assumption of a Gaussian distribution. Under a Poisson assumption, we fit a model to observed data using the negative log-likelihood score. We present a new algorithm for Poisson tensor factorization called CANDECOMP-PARAFAC Alternating Poisson Regression (CP-APR) that is based on a majorization-minimization approach. It can be shown that CP-APR is a generalization of the Lee-Seung multiplicative updates. We show how to prevent the algorithm from converging to non-KKT points and prove convergence of CP-APR under mild conditions. We also explain how to implement CP-APR for large-scale sparse tensors and present results on several data sets, both real and simulated.