On Separability of Covariance in Multiway Data Analysis

TL;DR

This paper investigates the theoretical limitations and computational challenges of representing multiway covariance matrices as sums of Kronecker products, revealing that such separability is generally hard to certify or approximate.

Contribution

It connects multiway covariance separability to quantum state separability, proving NP-hardness of finding optimal separable approximations and analyzing algorithmic convergence.

Findings

Most multiway covariances are not separable.

Determining the best separable approximation is NP-hard.

Standard algorithms often find near-optimal solutions.

Abstract

Multiway data analysis aims to uncover patterns in data structured as multi-indexed arrays, with multiway covariance playing a crucial role in many applications. However, the high dimensionality of multiway covariance presents significant computational challenges. To overcome these challenges, factorized covariance models have been proposed that rely on a separability assumption: the multiway covariance can be accurately expressed as a sum of Kronecker products of mode-wise covariances. This paper addresses the representability, certification, and approximation of such separable models, leaving statistical estimation or finite-sample properties aside. We reduce the question of whether a given covariance can be decomposed into a separable multiway form to an equivalent question about the separability of quantum states. Leveraging results from quantum information theory, we show that generic multiway covariances are typically \emph{not} separable and that determining the best separable approximation is NP-hard. These findings suggest that factorized covariance models can be overly restrictive and difficult to fit without additional structural assumptions. Nevertheless, our numerical experiments indicate that standard iterative algorithms, namely Frank-Wolfe and gradient descent, often converge close to the best separable approximation. As NP-hardness concerns worst-case computational complexity, Kronecker-separable approximations to multiway covariance could still be tractable to apply for analyzing many real-world datasets.