Recurrent Graph Tensor Networks: A Low-Complexity Framework for Modelling High-Dimensional Multi-Way Sequence

TL;DR

This paper introduces Recurrent Graph Tensor Networks, a novel low-complexity framework that combines tensor networks and graph filters to effectively model high-dimensional multi-way sequences, outperforming traditional RNNs.

Contribution

It proposes a new RNN architecture integrating tensor networks and graph filters to handle high-dimensional data efficiently, reducing parameter complexity and mitigating the Curse of Dimensionality.

Findings

Outperforms standard RNNs on multi-way sequence tasks

Reduces parameter complexity compared to traditional RNNs

Mitigates the Curse of Dimensionality in sequence modelling

Abstract

Recurrent Neural Networks (RNNs) are among the most successful machine learning models for sequence modelling, but tend to suffer from an exponential increase in the number of parameters when dealing with large multidimensional data. To this end, we develop a multi-linear graph filter framework for approximating the modelling of hidden states in RNNs, which is embedded in a tensor network architecture to improve modelling power and reduce parameter complexity, resulting in a novel Recurrent Graph Tensor Network (RGTN). The proposed framework is validated through several multi-way sequence modelling tasks and benchmarked against traditional RNNs. By virtue of the domain aware information processing of graph filters and the expressive power of tensor networks, we show that the proposed RGTN is capable of not only out-performing standard RNNs, but also mitigating the Curse of Dimensionality associated with traditional RNNs, demonstrating superior properties in terms of performance and complexity.