Scalable Bayesian Tensor Ring Factorization for Multiway Data Analysis

TL;DR

This paper introduces a scalable Bayesian tensor ring factorization model that effectively handles large-scale, multiway data, including discrete types, by integrating nonparametric priors, Pólya-Gamma augmentation, and efficient inference algorithms.

Contribution

It proposes a novel Bayesian tensor ring model with a nonparametric prior, Pólya-Gamma augmentation for discrete data, and scalable inference methods, overcoming previous limitations.

Findings

Outperforms existing methods in accuracy and scalability

Efficiently handles large-scale and discrete tensor data

Demonstrates superior performance on real-world datasets

Abstract

Tensor decompositions play a crucial role in numerous applications related to multi-way data analysis. By employing a Bayesian framework with sparsity-inducing priors, Bayesian Tensor Ring (BTR) factorization offers probabilistic estimates and an effective approach for automatically adapting the tensor ring rank during the learning process. However, previous BTR method employs an Automatic Relevance Determination (ARD) prior, which can lead to sub-optimal solutions. Besides, it solely focuses on continuous data, whereas many applications involve discrete data. More importantly, it relies on the Coordinate-Ascent Variational Inference (CAVI) algorithm, which is inadequate for handling large tensors with extensive observations. These limitations greatly limit its application scales and scopes, making it suitable only for small-scale problems, such as image/video completion. To address these issues, we propose a novel BTR model that incorporates a nonparametric Multiplicative Gamma Process (MGP) prior, known for its superior accuracy in identifying latent structures. To handle discrete data, we introduce the P\'olya-Gamma augmentation for closed-form updates. Furthermore, we develop an efficient Gibbs sampler for consistent posterior simulation, which reduces the computational complexity of previous VI algorithm by two orders, and an online EM algorithm that is scalable to extremely large tensors. To showcase the advantages of our model, we conduct extensive experiments on both simulation data and real-world applications.