Soft Tensor Regression

TL;DR

This paper introduces a Bayesian soft tensor regression framework that enhances flexibility and accuracy in modeling tensor predictors, overcoming limitations of traditional low-rank approximations and improving inference and prediction in complex data.

Contribution

The paper proposes a novel soft PARAFAC-based tensor regression method that allows variation around an overall mean, providing better estimation and theoretical guarantees regardless of tensor rank.

Findings

Soft tensor regression improves estimation accuracy.

The method achieves more precise variable selection.

Posterior distribution is weakly consistent regardless of tensor rank.

Abstract

Statistical methods relating tensor predictors to scalar outcomes in a regression model generally vectorize the tensor predictor and estimate the coefficients of its entries employing some form of regularization, use summaries of the tensor covariate, or use a low dimensional approximation of the coefficient tensor. However, low rank approximations of the coefficient tensor can suffer if the true rank is not small. We propose a tensor regression framework which assumes a soft version of the parallel factors (PARAFAC) approximation. In contrast to classic PARAFAC, where each entry of the coefficient tensor is the sum of products of row-specific contributions across the tensor modes, the soft tensor regression (Softer) framework allows the row-specific contributions to vary around an overall mean. We follow a Bayesian approach to inference, and show that softening the PARAFAC increases model flexibility, leads to improved estimation of coefficient tensors, more accurate identification of important predictor entries, and more precise predictions, even for a low approximation rank. From a theoretical perspective, we show that employing Softer leads to a weakly consistent posterior distribution of the coefficient tensor, irrespective of the true or approximation tensor rank, a result that is not true when employing the classic PARAFAC for tensor regression. In the context of our motivating application, we adapt Softer to symmetric and semi-symmetric tensor predictors and analyze the relationship between brain network characteristics and human traits.soft