Understanding the Kronecker Matrix-Vector Complexity of Linear Algebra

TL;DR

This paper investigates the complexity of estimating properties of matrices using only Kronecker-structured vector queries, establishing exponential lower bounds and contrasting with non-Kronecker cases.

Contribution

It provides the first exponential lower bounds for property testing of matrices with Kronecker-structured queries, highlighting fundamental differences from non-Kronecker scenarios.

Findings

Exponential lower bounds on query complexity for trace and eigenvalue estimation.

Algorithms with small alphabet queries cannot efficiently test if a matrix is zero.

Kronecker-structured vectors have exponentially smaller inner products than non-Kronecker vectors.

Abstract

We study the computational model where we can access a matrix $\mathbf{A}$ only by computing matrix-vector products $\mathbf{A}\mathrm{x}$ for vectors of the form $\mathrm{x} = \mathrm{x}_1 \otimes \cdots \otimes \mathrm{x}_q$. We prove exponential lower bounds on the number of queries needed to estimate various properties, including the trace and the top eigenvalue of $\mathbf{A}$. Our proofs hold for all adaptive algorithms, modulo a mild conditioning assumption on the algorithm's queries. We further prove that algorithms whose queries come from a small alphabet (e.g., $\mathrm{x}_i \in \{\pm1\}^n$) cannot test if $\mathbf{A}$ is identically zero with polynomial complexity, despite the fact that a single query using Gaussian vectors solves the problem with probability 1. In steep contrast to the non-Kronecker case, this shows that sketching $\mathbf{A}$ with different distributions of the same subguassian norm can yield exponentially different query complexities. Our proofs follow from the observation that random vectors with Kronecker structure have exponentially smaller inner products than their non-Kronecker counterparts.