Sample, computation vs storage tradeoffs for classification using tensor subspace models

TL;DR

This paper explores the tradeoffs between sample size, computation, and storage in supervised classification, proposing hierarchical tensor subspace models to improve efficiency and prevent overfitting.

Contribution

It introduces hierarchical Kronecker structured tensor subspaces for data embedding, enhancing tradeoffs and reducing overfitting in classification tasks.

Findings

Hierarchical tensor subspaces improve tradeoffs in classification.

Embedding with Kronecker structures prevents overfitting.

Hierarchical models outperform non-hierarchical counterparts.

Abstract

In this paper, we exhibit the tradeoffs between the (training) sample, computation and storage complexity for the problem of supervised classification using signal subspace estimation. Our main tool is the use of tensor subspaces, i.e. subspaces with a Kronecker structure, for embedding the data into lower dimensions. Among the subspaces with a Kronecker structure, we show that using subspaces with a hierarchical structure for representing data leads to improved tradeoffs. One of the main reasons for the improvement is that embedding data into these hierarchical Kronecker structured subspaces prevents overfitting at higher latent dimensions.