Bayesian Tensor-on-Tensor Regression with Efficient Computation

TL;DR

This paper introduces a Bayesian tensor-on-tensor regression method that estimates model dimension and parameters simultaneously, using efficient MCMC and optimization algorithms, with demonstrated superior performance and practical applications.

Contribution

It develops a novel Bayesian framework with algorithms for joint estimation of model dimension and parameters in tensor regression.

Findings

Superior performance in simulation studies

Effective uncertainty quantification

Successful application to real-world datasets

Abstract

We propose a Bayesian tensor-on-tensor regression approach to predict a multidimensional array (tensor) of arbitrary dimensions from another tensor of arbitrary dimensions, building upon the Tucker decomposition of the regression coefficient tensor. Traditional tensor regression methods making use of the Tucker decomposition either assume the dimension of the core tensor to be known or estimate it via cross-validation or some model selection criteria. However, no existing method can simultaneously estimate the model dimension (the dimension of the core tensor) and other model parameters. To fill this gap, we develop an efficient Markov Chain Monte Carlo (MCMC) algorithm to estimate both the model dimension and parameters for posterior inference. Besides the MCMC sampler, we also develop an ultra-fast optimization-based computing algorithm wherein the maximum a posteriori estimators for parameters are computed, and the model dimension is optimized via a simulated annealing algorithm. The proposed Bayesian framework provides a natural way for uncertainty quantification. Through extensive simulation studies, we evaluate the proposed Bayesian tensor-on-tensor regression model and show its superior performance compared to alternative methods. We also demonstrate its practical effectiveness by applying it to two real-world datasets, including facial imaging data and 3D motion data.