Residual Tensor Train: A Quantum-inspired Approach for Learning Multiple Multilinear Correlations

TL;DR

The paper introduces Residual Tensor Train (ResTT), a quantum-inspired multilinear model that captures complex feature correlations, improves training stability, and outperforms existing models on image and limited data classification tasks.

Contribution

ResTT is a novel quantum-inspired tensor network model that unifies multiple multilinear correlations and offers a relaxed weight initialization rule for stable training.

Findings

ResTT outperforms state-of-the-art tensor network and deep learning models on MNIST and Fashion-MNIST.

ResTT achieves better results than other statistical methods on complex, limited-data problems.

The model effectively captures multiple multilinear correlations from low to high orders.

Abstract

States of quantum many-body systems are defined in a high-dimensional Hilbert space, where rich and complex interactions among subsystems can be modelled. In machine learning, complex multiple multilinear correlations may also exist within input features. In this paper, we present a quantum-inspired multilinear model, named Residual Tensor Train (ResTT), to capture the multiple multilinear correlations of features, from low to high orders, within a single model. ResTT is able to build a robust decision boundary in a high-dimensional space for solving fitting and classification tasks. In particular, we prove that the fully-connected layer and the Volterra series can be taken as special cases of ResTT. Furthermore, we derive the rule for weight initialization that stabilizes the training of ResTT based on a mean-field analysis. We prove that such a rule is much more relaxed than that of TT, which means ResTT can easily address the vanishing and exploding gradient problem that exists in the existing TT models. Numerical experiments demonstrate that ResTT outperforms the state-of-the-art tensor network and benchmark deep learning models on MNIST and Fashion-MNIST datasets. Moreover, ResTT achieves better performance than other statistical methods on two practical examples with limited data which are known to have complex feature interactions.