Convex Tensor Decomposition via Structured Schatten Norm Regularization

TL;DR

This paper introduces structured Schatten norms for tensor decomposition, analyzes the performance of the latent approach, and confirms theoretical predictions with numerical simulations, advancing convex tensor decomposition methods.

Contribution

It provides a theoretical analysis of the latent structured Schatten norm approach, establishing its advantages and properties in convex tensor decomposition.

Findings

Latent approach performs as well as the best mode when the tensor is low-rank in that mode.

Theoretical prediction of mean squared error scaling matches numerical simulations.

Duality and consistency results for structured Schatten norms are established.

Abstract

We discuss structured Schatten norms for tensor decomposition that includes two recently proposed norms ("overlapped" and "latent") for convex-optimization-based tensor decomposition, and connect tensor decomposition with wider literature on structured sparsity. Based on the properties of the structured Schatten norms, we mathematically analyze the performance of "latent" approach for tensor decomposition, which was empirically found to perform better than the "overlapped" approach in some settings. We show theoretically that this is indeed the case. In particular, when the unknown true tensor is low-rank in a specific mode, this approach performs as good as knowing the mode with the smallest rank. Along the way, we show a novel duality result for structures Schatten norms, establish the consistency, and discuss the identifiability of this approach. We confirm through numerical simulations that our theoretical prediction can precisely predict the scaling behavior of the mean squared error.