Nonparametric Sparse Tensor Factorization with Hierarchical Gamma Processes

TL;DR

This paper introduces a nonparametric tensor factorization method using hierarchical Gamma processes that effectively models sparse observed tensors, capturing structural and relational information with enhanced interpretability and flexibility.

Contribution

It proposes a novel nonparametric tensor factorization framework leveraging hierarchical Gamma processes and Gaussian processes, providing interpretability and asymptotic sparsity guarantees.

Findings

Outperforms existing methods on benchmark datasets.

Effectively models sparse tensor data with structural and relational insights.

Provides a scalable stochastic variational inference algorithm.

Abstract

We propose a nonparametric factorization approach for sparsely observed tensors. The sparsity does not mean zero-valued entries are massive or dominated. Rather, it implies the observed entries are very few, and even fewer with the growth of the tensor; this is ubiquitous in practice. Compared with the existent works, our model not only leverages the structural information underlying the observed entry indices, but also provides extra interpretability and flexibility -- it can simultaneously estimate a set of location factors about the intrinsic properties of the tensor nodes, and another set of sociability factors reflecting their extrovert activity in interacting with others; users are free to choose a trade-off between the two types of factors. Specifically, we use hierarchical Gamma processes and Poisson random measures to construct a tensor-valued process, which can freely sample the two types of factors to generate tensors and always guarantees an asymptotic sparsity. We then normalize the tensor process to obtain hierarchical Dirichlet processes to sample each observed entry index, and use a Gaussian process to sample the entry value as a nonlinear function of the factors, so as to capture both the sparse structure properties and complex node relationships. For efficient inference, we use Dirichlet process properties over finite sample partitions, density transformations, and random features to develop a stochastic variational estimation algorithm. We demonstrate the advantage of our method in several benchmark datasets.