Factor Augmented Tensor-on-Tensor Neural Networks

TL;DR

This paper introduces FATTNN, a neural network that combines tensor factor models with deep learning to improve prediction accuracy and computational efficiency for tensor-on-tensor regression tasks involving complex, multi-dimensional data.

Contribution

The paper proposes a novel tensor neural network model that integrates tensor factorization to handle nonlinearity and reduce data dimensionality, outperforming traditional methods.

Findings

Significant improvement in prediction accuracy over benchmarks

Substantial reduction in computational time

Effective handling of nonlinear relationships in tensor data

Abstract

This paper studies the prediction task of tensor-on-tensor regression in which both covariates and responses are multi-dimensional arrays (a.k.a., tensors) across time with arbitrary tensor order and data dimension. Existing methods either focused on linear models without accounting for possibly nonlinear relationships between covariates and responses, or directly employed black-box deep learning algorithms that failed to utilize the inherent tensor structure. In this work, we propose a Factor Augmented Tensor-on-Tensor Neural Network (FATTNN) that integrates tensor factor models into deep neural networks. We begin with summarizing and extracting useful predictive information (represented by the ``factor tensor'') from the complex structured tensor covariates, and then proceed with the prediction task using the estimated factor tensor as input of a temporal convolutional neural network. The proposed methods effectively handle nonlinearity between complex data structures, and improve over traditional statistical models and conventional deep learning approaches in both prediction accuracy and computational cost. By leveraging tensor factor models, our proposed methods exploit the underlying latent factor structure to enhance the prediction, and in the meantime, drastically reduce the data dimensionality that speeds up the computation. The empirical performances of our proposed methods are demonstrated via simulation studies and real-world applications to three public datasets. Numerical results show that our proposed algorithms achieve substantial increases in prediction accuracy and significant reductions in computational time compared to benchmark methods.