Mode-Aware Non-Linear Tucker Autoencoder for Tensor-based Unsupervised Learning

TL;DR

This paper introduces MA-NTAE, a non-linear tensor autoencoder that efficiently captures structural information in high-dimensional tensor data, outperforming traditional methods in compression and clustering tasks.

Contribution

It presents a novel non-linear Tucker autoencoder with a flexible mode-aware encoding strategy, improving scalability and effectiveness for high-order tensor data.

Findings

MA-NTAE outperforms standard autoencoders in compression tasks.

It achieves better clustering results on high-dimensional tensors.

Computational complexity grows linearly with tensor order.

Abstract

High-dimensional data, particularly in the form of high-order tensors, presents a major challenge in self-supervised learning. While MLP-based autoencoders (AE) are commonly employed, their dependence on flattening operations exacerbates the curse of dimensionality, leading to excessively large model sizes, high computational overhead, and challenging optimization for deep structural feature capture. Although existing tensor networks alleviate computational burdens through tensor decomposition techniques, most exhibit limited capability in learning non-linear relationships. To overcome these limitations, we introduce the Mode-Aware Non-linear Tucker Autoencoder (MA-NTAE). MA-NTAE generalized classical Tucker decomposition to a non-linear framework and employs a Pick-and-Unfold strategy, facilitating flexible per-mode encoding of high-order tensors via recursive unfold-encode-fold operations, effectively integrating tensor structural priors. Notably, MA-NTAE exhibits linear growth in computational complexity with tensor order and proportional growth with mode dimensions. Extensive experiments demonstrate MA-NTAE's performance advantages over standard AE and current tensor networks in compression and clustering tasks, which become increasingly pronounced for higher-order, higher-dimensional tensors.