Tensor Reconstruction Beyond Constant Rank

TL;DR

This paper introduces new algorithms for tensor reconstruction, including the first efficient method for tensor rank determination and optimal tensor decomposition from black-box access, advancing the understanding of depth-3 circuit polynomial reconstruction.

Contribution

It provides the first efficient algorithms for tensor rank and decomposition from black-box access, correcting previous errors and introducing rank-preserving coordinate subspace learning techniques.

Findings

First efficient tensor rank algorithm from black-box access

Optimal tensor decomposition for super-constant rank tensors

Correction of prior errors in tensor circuit analysis

Abstract

We give reconstruction algorithms for subclasses of depth-3 arithmetic circuits. In particular, we obtain the first efficient algorithm for finding tensor rank, and an optimal tensor decomposition as a sum of rank-one tensors, when given black-box access to a tensor of super-constant rank. We obtain the following results: 1. A deterministic algorithm that reconstructs polynomials computed by $\Sigma^{[k]}\bigwedge^{[d]}\Sigma$ circuits in time $\mathsf{poly}(n,d,c) \cdot \mathsf{poly}(k)^{k^{k^{10}}}$ 2. A randomized algorithm that reconstructs polynomials computed by multilinear $\Sigma^{k]}\prod^{[d]}\Sigma$ circuits in time $\mathsf{poly}(n,d,c) \cdot k^{k^{k^{k^{O(k)}}}}$ 3. A randomized algorithm that reconstructs polynomials computed by set-multilinear $\Sigma^{k]}\prod^{[d]}\Sigma$ circuits in time $\mathsf{poly}(n,d,c) \cdot k^{k^{k^{k^{O(k)}}}}$, where $c=\log q$ if $\mathbb{F}=\mathbb{F}_q$ is a finite field, and $c$ equals the maximum bit complexity of any coefficient of $f$ if $\mathbb{F}$ is infinite. Prior to our work, polynomial time algorithms for the case when the rank, $k$, is constant, were given by Bhargava, Saraf and Volkovich [BSV21]. Another contribution of this work is correcting an error from a paper of Karnin and Shpilka [KS09] that affected Theorem 1.6 of [BSV21]. Consequently, the results of [KS09, BSV21] continue to hold, with a slightly worse setting of parameters. For fixing the error we study the relation between syntactic and semantic ranks of $\Sigma\Pi\Sigma$ circuits. We obtain our improvement by introducing a technique for learning rank preserving coordinate-subspaces. [KS09] and [BSV21] tried all choices of finding the "correct" coordinates, which led to having a fast growing function of $k$ at the exponent of $n$. We find these spaces in time that is growing fast with $k$, yet it is only a fixed polynomial in $n$.