Matrix Product Operators for Sequence to Sequence Learning

TL;DR

This paper introduces a novel sequence-to-sequence machine learning model based on matrix product operators, demonstrating its effectiveness on cellular automata, coupled maps, and classification tasks, with advantages over traditional neural networks.

Contribution

The paper presents a new matrix product operator-based model for sequence prediction, extending quantum physics tools to machine learning with demonstrated flexibility and improved performance.

Findings

Exact solutions for cellular automata using matrix product operators

Outperforms conditional random fields and LSTM networks in sequence prediction

Can be adapted for classification tasks

Abstract

The method of choice to study one-dimensional strongly interacting many body quantum systems is based on matrix product states and operators. Such method allows to explore the most relevant, and numerically manageable, portion of an exponentially large space. It also allows to describe accurately correlations between distant parts of a system, an important ingredient to account for the context in machine learning tasks. Here we introduce a machine learning model in which matrix product operators are trained to implement sequence to sequence prediction, i.e. given a sequence at a time step, it allows one to predict the next sequence. We then apply our algorithm to cellular automata (for which we show exact analytical solutions in terms of matrix product operators), and to nonlinear coupled maps. We show advantages of the proposed algorithm when compared to conditional random fields and bidirectional long short-term memory neural network. To highlight the flexibility of the algorithm, we also show that it can readily perform classification tasks.