Learning Relevant Features of Data with Multi-scale Tensor Networks

TL;DR

This paper introduces layered tree tensor networks inspired by physics for data feature extraction, demonstrating efficient unsupervised and supervised learning on image datasets with promising results.

Contribution

It adapts coarse-graining tensor network algorithms for data analysis, combining unsupervised and supervised methods for effective feature learning.

Findings

Linear scaling with input and dataset size

High accuracy on MNIST and fashion-MNIST

Effective feature reduction with prior knowledge

Abstract

Inspired by coarse-graining approaches used in physics, we show how similar algorithms can be adapted for data. The resulting algorithms are based on layered tree tensor networks and scale linearly with both the dimension of the input and the training set size. Computing most of the layers with an unsupervised algorithm, then optimizing just the top layer for supervised classification of the MNIST and fashion-MNIST data sets gives very good results. We also discuss mixing a prior guess for supervised weights together with an unsupervised representation of the data, yielding a smaller number of features nevertheless able to give good performance.