Embeddings of Schatten Norms with Applications to Data Streams

TL;DR

This paper investigates how to embed Schatten norms of matrices into lower-dimensional spaces via linear transformations, providing nearly tight bounds on the size of such embeddings and applying these results to data stream algorithms for norm approximation.

Contribution

It nearly resolves the embeddability problem for Schatten norms with linear maps of the form R·A·S, offering tight bounds and addressing an open question on space-approximation trade-offs in data streams.

Findings

Provided nearly matching upper and lower bounds for embeddings of Schatten norms.

Upper bounds are oblivious, independent of the matrices, while lower bounds hold even with matrix-dependent transformations.

Achieved improved space complexity for Schatten-1 norm estimation in data streams.

Abstract

Given an $n \times d$ matrix $A$, its Schatten-$p$ norm, $p \geq 1$, is defined as $\|A\|_p = \left (\sum_{i=1}^{\textrm{rank}(A)}\sigma_i(A)^p \right )^{1/p}$, where $\sigma_i(A)$ is the $i$-th largest singular value of $A$. These norms have been studied in functional analysis in the context of non-commutative $\ell_p$-spaces, and recently in data stream and linear sketching models of computation. Basic questions on the relations between these norms, such as their embeddability, are still open. Specifically, given a set of matrices $A^1, \ldots, A^{\operatorname{poly}(nd)} \in \mathbb{R}^{n \times d}$, suppose we want to construct a linear map $L$ such that $L(A^i) \in \mathbb{R}^{n' \times d'}$ for each $i$, where $n' \leq n$ and $d' \leq d$, and further, $\|A^i\|_p \leq \|L(A^i)\|_q \leq D_{p,q} \|A^i\|_p$ for a given approximation factor $D_{p,q}$ and real number $q \geq 1$. Then how large do $n'$ and $d'$ need to be as a function of $D_{p,q}$? We nearly resolve this question for every $p, q \geq 1$, for the case where $L(A^i)$ can be expressed as $R \cdot A^i \cdot S$, where $R$ and $S$ are arbitrary matrices that are allowed to depend on $A^1, \ldots, A^t$, that is, $L(A^i)$ can be implemented by left and right matrix multiplication. Namely, for every $p, q \geq 1$, we provide nearly matching upper and lower bounds on the size of $n'$ and $d'$ as a function of $D_{p,q}$. Importantly, our upper bounds are {\it oblivious}, meaning that $R$ and $S$ do not depend on the $A^i$, while our lower bounds hold even if $R$ and $S$ depend on the $A^i$. As an application of our upper bounds, we answer a recent open question of Blasiok et al. about space-approximation trade-offs for the Schatten $1$-norm, showing in a data stream it is possible to estimate the Schatten-$1$ norm up to a factor of $D \geq 1$ using $\tilde{O}(\min(n,d)^2/D^4)$ space.