Barriers for fast matrix multiplication from irreversibility

TL;DR

This paper introduces a new barrier concept called irreversibility, which limits the effectiveness of current methods in reducing the matrix multiplication exponent below 2, by analyzing the irreversibility of tensors used in these approaches.

Contribution

The paper defines tensor irreversibility and proves it imposes fundamental limits on improving the matrix multiplication exponent, extending previous barrier results.

Findings

Irreversibility bounds for key tensors like Coppersmith--Winograd tensors

Any approach using an irreversible tensor cannot achieve ω=2

Quantum and support functionals provide lower bounds on irreversibility

Abstract

Determining the asymptotic algebraic complexity of matrix multiplication, succinctly represented by the matrix multiplication exponent $\omega$, is a central problem in algebraic complexity theory. The best upper bounds on $\omega$, leading to the state-of-the-art $\omega \leq 2.37..$, have been obtained via the laser method of Strassen and its generalization by Coppersmith and Winograd. Recent barrier results show limitations for these and related approaches to improve the upper bound on $\omega$. We introduce a new and more general barrier, providing stronger limitations than in previous work. Concretely, we introduce the notion of "irreversibility" of a tensor and we prove (in some precise sense) that any approach that uses an irreversible tensor in an intermediate step (e.g., as a starting tensor in the laser method) cannot give $\omega = 2$. In quantitative terms, we prove that the best upper bound achievable is lower bounded by two times the irreversibility of the intermediate tensor. The quantum functionals and Strassen support functionals give (so far, the best) lower bounds on irreversibility. We provide lower bounds on the irreversibility of key intermediate tensors, including the small and big Coppersmith--Winograd tensors, that improve limitations shown in previous work. Finally, we discuss barriers on the group-theoretic approach in terms of "monomial" irreversibility.

Equations

$$\text{multiplying numbers}\stackrel{\omega }{\to }\text{multiplying matrices}.$$

$$\text{multiplying numbers}\stackrel{{\alpha }_1}{\to }\text{intermediate problem}\stackrel{{\alpha }_2}{\to }\text{multiplying matrices}.$$

$$\begin{gathered} \text{multiplying numbers}\stackrel{{\alpha }_1}{\to }\text{intermediate problem}\stackrel{{\alpha }_2}{\to }\text{multiplying matrices} \\ \stackrel{1/2}{\to }\text{multiplying numbers}. \end{gathered}$$

$${\mathrm{cw}}_q=\sum _{i=1}^q{e}_{0,i,i}+e_{i,0,i}+e_{i,i,0}$$

$${\mathrm{CW}}_q=e_{0,0,q+1}+e_{0,q+1,0}+e_{q+1,0,0}+\sum _{i=1}^q{e}_{0,i,i}+e_{i,0,i}+e_{i,i,0}$$

$$t_n=\sum _{\begin{array}{c}i,j,k=0\text{mathchar}58\\ i+j=k\end{array}}^{n-1}{e}_{i,j,k}$$

$$q$$

$$t=(t_{i_1,i_2,i_3}{)}_{i_1\in [n_1],i_2\in [n_1],i_3\in [n_1]}$$

$$\mathrm{supp}(t):=\{(i_1,i_2,i_3)\in [n_1]\times [n_2]\times [n_3]\text{mathchar}58t_{i_1,i_2,i_3}\ne 0\}.$$

$$((A_1,A_2,A_3)\cdot t{)}_{j_1,j_2,j_3}=\sum _{i_1,i_2,i_3}(A_1{)}_{j_1,i_1}(A_2{)}_{j_2,i_2}(A_3{)}_{j_3,i_3}{t}_{i_1,i_2,i_3}.$$

$$\text{underaccent}\text{wtilde}\mathrm{R}(t):=\underset{n\to \infty }{\lim }R(t^{\otimes n}{)}^{1/n}=\underset{n}{\inf }R(t^{\otimes n}{)}^{1/n}$$

$$\text{underaccent}\text{wtilde}\mathrm{Q}(t):=\underset{n\to \infty }{\lim }Q(t^{\otimes n}{)}^{1/n}=\underset{n}{\sup }Q(t^{\otimes n}{)}^{1/n}.$$

$$⟨a,b,c⟩:=\sum _{i=1}^a\sum _{j=1}^b\sum _{k=1}^c{e}_{(i,j),(j,k),(k,i)}\in {\mathbb{F}}^{ab\times bc\times ca}.$$

$$\text{underaccent}\text{wtilde}\mathrm{Q}(⟨h,h,h⟩)=h^2.$$

$$E=\{((i,j),(j,k),(k,i))\text{mathchar}58i,j,k\in [p]\}.$$

$$W_1=\{(x,x+a)\in V_1\text{mathchar}58x\in [p],a\in A\}$$

$$W_2=\{(x,x+a)\in V_2\text{mathchar}58x\in [p],a\in A\}$$

$$W_3=\{(x+2a,x)\in V_3\text{mathchar}58x\in [p],a\in A\}.$$

$$F=\{((i,j),(j,k),(k,i))\text{mathchar}58(i,j,k)=(x,x+a,x+2a)\text{ for some }x\in [p]\text{ and }a\in A\}$$

$$(i,j,k)=(x,x+a,x+a+b=x+2c)$$

$$\omega (t,s)$$

$$=\underset{n}{\sup }\frac{1}{n}\min \{m\in {\mathbb{N}}_{\ge 1}\text{mathchar}58t^{\otimes m}\ge s^{\otimes n}\}.$$

$$\omega (⟨2⟩,t)={\log }_2\text{underaccent}\text{wtilde}\mathrm{R}(t)$$

$$\omega (t,⟨2⟩)=1/({\log }_2\text{underaccent}\text{wtilde}\mathrm{Q}(t))$$

$$\omega (⟨2⟩,⟨2,2,2⟩)=\omega .$$

$$\omega (⟨2,2,2⟩,⟨2⟩)=\frac{1}{2}.$$

$$i(t):=\omega (⟨2⟩,t)\omega (t,⟨2⟩).$$

$$i(t)=\frac{{\log }_2\text{underaccent}\text{wtilde}\mathrm{R}(t)}{{\log }_2\text{underaccent}\text{wtilde}\mathrm{Q}(t)}.$$

$$i(t)\ge 1.$$

$${\mathbf{i}}_{\mathrm{M}}(t):=\omega (⟨2⟩,t){\omega }_{\mathrm{M}}(t,⟨2⟩).$$

$${\mathbf{i}}_{\mathrm{M}}(t)=\frac{{\log }_2\text{underaccent}\text{wtilde}\mathrm{R}(t)}{{\log }_2\text{underaccent}\text{wtilde}{\mathrm{Q}}_{\mathrm{M}}(t)}.$$

$$⟨G{⟩}_{\mathbb{F}}:=\sum _{g,h\in G}{e}_{g,h,gh}.$$

Full text

We introduce a new and more general barrier, providing stronger limitations than in previous work. Concretely, we introduce the notion of irreversibility of a tensor, and we prove (in some precise sense) that any approach that uses an irreversible tensor in an intermediate step (e.g., as a starting tensor in the laser method) cannot give $\omega=2$. In quantitative terms, we prove that the best upper bound achievable is lower bounded by twice the irreversibility of the intermediate tensor. The quantum functionals and Strassen support functionals give (so far, the best) lower bounds on irreversibility. We provide lower bounds on the irreversibility of key intermediate tensors, including the small and big Coppersmith–Winograd tensors, that improve limitations shown in previous work. Finally, we discuss barriers on the group-theoretic approach in terms of monomial irreversibility.

1. 1 Introduction
2. 2 Irreversibility of tensors
3. 3 Irreversibility implies barriers
4. 4 Methods for lower bounding irreversibility
5. 5 Explicit barriers
6. 6 Outlook for further barriers
7. A Code to verify the numerical examples

1. Introduction

1.1.

Determining the asymptotic algebraic complexity of matrix multiplication is a central open problem in algebraic complexity theory. Several methods for constructing fast matrix multiplication algorithms have been developed, but at a high level they typically consist of two parts: an efficient reduction of matrix multiplication to an intermediate problem (some bilinear map, i.e. some 3-tensor) and an efficient algorithm for the intermediate problem. Recent results have shown barriers for such constructions to yield fast matrix multiplication algorithms [AFLG15, BCC*+*17a, BCC*+*17b, AW18a, AW18b]. We give a barrier, based on a new notion called irreversibility, that is more general and in some cases stronger than the barriers from previous work.

1.2. Matrix multiplication barriers

The matrix multiplication exponent $\omega$ is defined as the infimum over all real numbers $\beta$ such that any two $n\times n$ matrices can be multiplied with $\mathcal{O}(n^{\beta})$ algebraic operations, and thus $\omega$ represents the asymptotic algebraic complexity of matrix multiplication. The bounds $2\leq\omega\leq 3$ hold trivially. Strassen published the first non-trivial upper bound $\omega\leq\log_{2}7$ in 1969 [Str69]. In the decades that followed, through the development of several ingenious methods by various people, the upper bound was improved to the state-of-the-art bound $\omega\leq 2.37..$, and the pursuit to prove whether $\omega=2$ or $\omega>2$ has been ongoing [CW90, Sto10, Wil12, LG14, CU03, CU13]. As mentioned before, these upper bound methods typically consist of a reduction of matrix multiplication to an intermediate problem and an efficient algorithm for the intermediate problem.

Ambainis, Filmus and Le Gall [AFLG15], for the first time, proved a barrier result for some collection of such methods. Namely, they showed that a variety of methods that go via the big Coppersmith–Winograd tensor as an intermediate problem cannot give $\omega=2$, and in fact not even $\omega\leq 2.30..$. We call any lower bound for all upper bounds on $\omega$ that can be obtained by some method, a barrier for that method. In general, barriers in the sense of limitations to proof methods have a long history in computational complexity theory and recognizing barriers is a natural step towards finding proof methods that do solve the problem at hand.

Next, Alman and Williams [AW18a, AW18b] extended the realm of barriers beyond the scope of the Ambainis et al. barrier, to a larger collection of methods. Also Blasiak, Church, Cohn, Grochow, Naslund, Sawin, and Umans [BCC*+*17a] and Blasiak, Church, Cohn, Grochow, and Umans [BCC*+*17b] studied barriers, namely barriers for a subset of the group-theoretic method. Both the Blasiak et al. and the Alman and Williams barriers rely on studying versions of asymptotic subrank of an intermediate problem.

We give a barrier that applies more generally than all previous barriers and that is in some cases stronger. Our barrier also relies on studying versions of asymptotic subrank, which together with the notion of asymptotic rank we combine into a single parameter called irreversibility. Our barrier simplifies and generalizes previous barriers and tightly connects the barrier literature to central notions from the framework of Strassen [Str87, Str88, Str91, CVZ18]. Alman [Alm19] reported very similar independent results shortly after our manuscript appeared on the arXiv, which we jointly presented at the Computational Complexity Conference 2019 in New Brunswick. For all the tensors mentioned, the barriers of Alman are identical to ours. A subtle difference between the works is that Alman considers asymptotic slice rank instead of asymptotic subrank.

1.3. Our barrier: informal explanation

Our barrier relies on two ideas: (i) we think of computational problems as resources that can be reduced to one another and (ii) such reductions satisfy a triangle inequality which limits the possible chains of reductions. An informal explanation of our barrier is as follows. The matrix multiplication exponent $\omega$ is the optimal rate at which the problem of multiplying matrices can be reduced to the problem of multiplying numbers, or in other words, the optimal rate at which the problem of multiplying numbers can be transformed into the problem of multiplying matrices,

[TABLE]

By this we mean that for any $n$ the problem of multiplying $n\times n$ matrices can be reduced to the problem of multiplying $n^{\omega+o(1)}$ pairs of numbers (and some additions, but they do not have an influence on the complexity). We will make these notions precise later. For now, we stick to the high-level picture. Rates of transformation naturally satisfy a triangle inequality. Therefore, upper bounds on $\omega$ can be obtained by combining the rate of transformation $\alpha_{1}$ from the problem of multiplying numbers to some intermediate problem and the rate of transformation $\alpha_{2}$ from the intermediate problem to the problem of multiplying matrices; this is the two-component approach alluded to earlier,

[TABLE]

That is, $\alpha_{1}\alpha_{2}\geq\omega$. We define the irreversibility of the intermediate tensor, roughly speaking, as the optimal rate of transformation from the problem of multiplying numbers to the intermediate problem and back to the problem of multiplying numbers. Strassen [Str88] (see Section 2.1) showed that the transformation rate from the matrix multiplication problem to the problem of multiplying numbers is $\tfrac{1}{2}$, so we can extend the chain in (2) to

[TABLE]

Using the triangle inequality again we see that $\alpha_{1}\alpha_{2}/2$ is at least the irreversibility of the intermediate problem, and hence the irreversibility of the intermediate problem provides limitations on the upper bounds $\alpha_{1}\alpha_{2}\geq\omega$ that can be obtained from (2). This is formalized in Theorem 17.

1.4. Explicit numerical barriers

To exemplify our barrier we show that the support functionals [Str91] and quantum functionals [CVZ18] give (so far, the best) lower bounds on the irreversibility of the following families of intermediate problems. These intermediate problems are encoded as tensors. A tensor is a 3-dimensional array of numbers. We use $e_{i,j,k}$ to denote the tensor that is zero everywhere except for a one in coordinate $(i,j,k)$. We will discuss tensors in more details later. The families of intermediate tensors we consider are:

• •

the small Coppersmith–Winograd tensors

[TABLE]

• •

the big Coppersmith–Winograd tensors

[TABLE]

• •

the reduced polynomial multiplication tensors

[TABLE]

Our irreversibility lower bounds lead to the following explicit barriers (Section 5.1 and Section 5.2), rounded to five decimal places.

[TABLE]

In Appendix A we provide code to compute the values in these tables to arbitrary precision and for arbitrary parameters $q$ and $n$. As suggested by the values in the above tables, both the $\mathrm{cw}_{q}$-barrier and $\mathrm{CW}_{q}$-barrier increase when $q$ grows, and converge to 3 when $q$ goes to infinity. The $t_{n}$-barrier, on the other hand, decreases when $n$ grows, and converges to $2$ when $n$ goes to infinity. Families of tensors for which the barrier converges to 2 in the limit, like $t_{n}$, may be crucial in proving that $\omega=2$.

We stress that our method can be applied to any tensor, not only the ones above. The tensors listed above play a special role in the literature and that is why we single them out. The small and big Coppersmith–Winograd tensors come from the paper [CW90]. They first analyze $\mathrm{cw}_{5}$ to get an upper bound on $\omega$, and then improve it by analyzing $\mathrm{CW}_{6}$ and its Kronecker square. All papers from then on (Stothers, Vassilevska–Williams, Le Gall) use powers of $\mathrm{CW}_{5}$ to get upper bounds on $\omega$. The reduced polynomial multiplication tensors $t_{n}$ give an example of a family of tensors for which the irreducibility goes to 1 when $n$ goes to infinity and hence the barrier becomes trivial in the limit.

1.5. Comparison and other applications

Compared to [AFLG15], whose barriers apply to the laser method, our barriers are valid for a larger class of approaches (and naturally we obtain lower barriers). Compared to [AW18b], whose barriers apply to what we call monomial degeneration, our barriers are valid for a larger class of approaches but our barriers are also higher. As a variation on our barrier we introduce a monomial version. Compared to [BCC*+*17a] and [BCC*+*17b] our monomial barriers are valid for a class of approaches that includes their simultaneous triple product property (STPP) approach, and thus we provide a uniform view on the barriers that have appeared in the literature. We have not tried to optimize the barriers that we obtain, but focus instead on introducing the barrier itself. The barrier in [Alm19] is very similar to ours, except for using asymptotic slice rank instead of asymptotic subrank.

It will become clear to the reader during the development of our ideas that they not only apply to the problem of fast matrix multiplication, but extend to give barriers for the more general problem of constructing fast rectangular matrix multiplication algorithms (see the follow-up results on barriers for rectangular matrix multiplication in [CGLZ20]) or even transformations between arbitrary powers of tensors. Such transformations may represent, for example, asymptotic SLOCC (stochastic local operations and classical communication) reductions among multipartite quantum states [BPR*+*00, DVC00, VDDMV02, HHHH09].

We define irreversibility in Section 2. In Section 3 we discuss how irreversibility implies a barrier. In Section 4 we discuss methods to analyze irreversibility. Finally, in Section 5 we exhibit explicit irreversibility barriers.

2. Irreversibility of tensors

We begin by introducing some standard terminology and results regarding tensors and matrix multiplication. Then we discuss a useful notion called the relative exponent of two tensors and we define the irreversibility of a tensor. After that we introduce the monomial versions of these ideas. Finally, we discuss what values the irreducibility can take.

2.1. Tensors and matrix multiplication

We begin with some standard terminology and results regarding tensors and matrix multiplication. Standard references for this material are [BCS97] and [Blä13]. Let $\mathbb{F}$ be a field. Everything we discuss works over all fields, except when mentioned otherwise. An $n_{1}\times n_{2}\times n_{3}$ tensor (over $\mathbb{F}$) is a three-dimensional array

[TABLE]

of field elements $t_{i_{1},i_{2},i_{3}}\in\mathbb{F}$. We denote the set of all $n_{1}\times n_{2}\times n_{3}$ tensors by $\mathbb{F}^{n_{1}\times n_{2}\times n_{3}}$. The support of $t$ is defined as the set

[TABLE]

We use $e_{i,j,k}$ to denote the tensor that is zero everywhere except for a one in coordinate $(i,j,k)$. For any $m_{i}\times n_{i}$ matrices $A_{i}$ over $\mathbb{F}$ we define the tensor $(A_{1},A_{2},A_{3})\cdot t\in\mathbb{F}^{m_{1}\times m_{2}\times m_{3}}$ by

[TABLE]

For $t\in\mathbb{F}^{n_{1}\times n_{2}\times n_{3}}$ and $s\in\mathbb{F}^{m_{1}\times m_{2}\times m_{3}}$ we write $t\geq s$ and say that $t$ restricts to $s$ if there are matrices $A_{i}$ such that $(A_{1},A_{2},A_{3})\cdot t=s$. We say $t$ and $s$ are equivalent if $t\geq s$ and $s\geq t$. We say $t$ and $s$ are isomorphic and write $s\cong t$ if $s=(A_{1},A_{2},A_{3})\cdot t$ for invertible matrices $A_{i}$. We will often simplify the notation and simply write $s=t$ when $s$ and $t$ are isomorphic.

For $n\in\mathbb{N}$ we define the rank-$n$ unit tensor $\langle n\rangle\coloneqq\sum_{i=1}^{n}e_{i,i,i}\in\mathbb{F}^{n\times n\times n}$. A simple tensor is a tensor that is an outer product $\sum_{i,j,k}u_{i}v_{j}w_{k}\,e_{i,j,k}$ of three vectors $u$, $v$ and $w$. The tensor rank of $t$ is defined as the smallest $r$ such that $t$ can be written as a sum of $r$ simple tensors, that is, $t=\sum_{\ell=1}^{r}\sum_{i,j,k}u^{(\ell)}_{i}v^{(\ell)}_{j}w^{(\ell)}_{k}\,e_{i,j,k}$ for vectors $u^{(\ell)}$, $v^{(\ell)}$ and $w^{(\ell)}$. It is crucial for us, but not hard to see, that we can phrase tensor rank in terms of the restriction preorder and diagonal tensors via $\operatorname{R}(t)=\min\{n\in\mathbb{N}\mathrel{\mathop{\mathchar 58\relax}}t\leq\langle n\rangle\}$. Indeed, if $t=(A_{1},A_{2},A_{3})\cdot\langle r\rangle$, then the columns of the matrices $A_{1}$, $A_{2}$ and $A_{3}$ provide the vectors $u^{(\ell)}$, $v^{(\ell)}$ and $w^{(\ell)}$ for a tensor rank upper bound, and in the other direction such vectors $u^{(\ell)}$, $v^{(\ell)}$ and $w^{(\ell)}$ provide the matrices $A_{1}$, $A_{2}$ and $A_{3}$. What is nice about this definition is that it naturally gives rise to the following tensor parameter, which is central for our barriers: The subrank of $t$ is defined as $\operatorname{Q}(t)\coloneqq\max\{n\in\mathbb{N}\mathrel{\mathop{\mathchar 58\relax}}\langle n\rangle\leq t\}$.

Extending the definition of Kronecker product for matrices, we define the Kronecker product $s\otimes t\in\mathbb{F}^{(m_{1}n_{2})\times(m_{2}n_{2})\times(m_{3}n_{3})}$ for tensors $s\in\mathbb{F}^{m_{1}\times m_{2}\times m_{3}}$ and $t\in\mathbb{F}^{n_{1}\times n_{2}\times n_{3}}$ as the tensor that has coefficients given by all pairwise products of coefficients of $s$ and $t$. Extending the definition of direct sum for matrices, we define the direct sum $s\oplus t\in\mathbb{F}^{(m_{1}+n_{2})\times(m_{2}+n_{2})\times(m_{3}+n_{3})}$ as the tensor obtained by placing $s$ and $t$ as blocks in a block-diagonal tensor. In long expressions we will often write $t^{n}$ to denote $t^{\otimes n}$. The product $\otimes$, the sum $\oplus$ and the diagonal tensors $\langle n\rangle$ behave well with respect to each other. One fact we will use frequently is that the direct sum of $n$ copies of a tensor $t$ is isomorphic to the Kronecker product of the diagonal tensor $\langle n\rangle$ and $t$.

For any tensor $t\in\mathbb{F}^{n_{1}\times n_{2}\times n_{3}}$ we define linear maps $t_{1}\mathrel{\mathop{\mathchar 58\relax}}\mathbb{F}^{n_{1}}\to\mathbb{F}^{n_{2}\times n_{3}}\mathrel{\mathop{\mathchar 58\relax}}v\mapsto\sum_{i,j,k}v_{i}t_{i,j,k}e_{j,k}$, $t_{2}\mathrel{\mathop{\mathchar 58\relax}}\mathbb{F}^{n_{2}}\to\mathbb{F}^{n_{1}\times n_{3}}\mathrel{\mathop{\mathchar 58\relax}}v\mapsto\sum_{i,j,k}v_{j}t_{i,j,k}e_{i,k}$ and $t_{3}\mathrel{\mathop{\mathchar 58\relax}}\mathbb{F}^{n_{3}}\to\mathbb{F}^{n_{1}\times n_{2}}\mathrel{\mathop{\mathchar 58\relax}}v\mapsto\sum_{i,j,k}v_{k}t_{i,j,k}e_{i,j}$. These are called the flattenings of $t$. We call each function $t\mapsto\operatorname{rk}(t_{i})$ a flattening rank of $t$. A basic observation and tool is that each flattening rank is multiplicative under the Kronecker product on tensors, additive under the direct sum on tensors, and monotone under the preorder $\leq$ on tensors.

The asymptotic rank of $t$ is defined as

[TABLE]

and the asymptotic subrank of $t$ is defined as

[TABLE]

We need to argue that the limits in the definitions exist. If $t\leq\langle n\rangle$ and $s\leq\langle m\rangle$, then we also have $t\otimes s\leq\langle nm\rangle$. It follows from this that tensor rank is sub-multiplicative under $\otimes$. Similarly, subrank is super-multiplicative under $\otimes$. Therefore, by Fekete’s lemma,111Fekete’s lemma is as follows. Let $r_{1},r_{2},r_{3},\ldots\in\mathbb{R}_{\geq 0}$ satisfy $r_{n+m}\leq r_{n}+r_{m}$ for all $n,m$. Then it holds that $\lim_{n\to\infty}r_{n}/n=\inf_{n}r_{n}/n$. See, e.g., [PS98, No. 98]. the above limits exist and equal the respective infimum and supremum. We note that asymptotic rank and asymptotic subrank may equivalently be defined using border rank and border subrank, respectively, which are the approximative versions of rank and subrank (e.g., see [Blä13]).

For $a,b,c\in\mathbb{N}_{\geq 1}$ the matrix multiplication tensor $\langle a,b,c\rangle$ is defined as

[TABLE]

To see how $\langle n,n,n\rangle$ encodes multiplication of $n\times n$ matrices, let $E_{i,j}$ be the $n\times n$ matrix that is all-zero except for a 1 at coordinate $(i,j)$, so that the $E_{i,j}$ form the standard basis of the vector space of $n\times n$ matrices. Note that $E_{i,j}E_{j,k}=E_{i,k}$ and that $\sum_{i=1}^{n}\sum_{j=1}^{n}\sum_{k=1}^{n}A_{(i,j)}B_{(j,k)}E_{i,k}=AB$. Thus suitable contractions of $\langle n,n,n\rangle$ with the matrices $A$ and $B$ produces the product $AB$. It is a standard result ([Blä13, Section 4]) that the tensor rank of $\langle n,n,n\rangle$ equals, up to a constant factor, the arithmetic complexity of multiplying two $n\times n$ matrices. The argument is roughly as follows. Tensor rank upper bounds lead directly to matrix multiplication algorithms by the aforementioned contractions. On the other hand, any arithmetic matrix multiplication algorithm can be made into an arithmetic circuit with an addition layer followed by a bilinear multiplication layer followed by an addition layer, in such a way that the number of bilinear multiplication nodes is at most twice the number of multiplications in the arithmetic algorithm. The blow-up by a factor of two is because we only allow bilinear multiplications in our arithmetic circuit. This representation leads directly to a tensor rank upper bound. As we said before, the matrix multiplication exponent $\omega$ is defined as the infimum over all $\beta$ for which the arithmetic complexity of multiplying two $n\times n$ matrices is at most $\mathcal{O}(n^{\beta})$. Matrix multiplication tensors are multiplicative in the sense that $\langle a,b,c\rangle\otimes\langle d,e,f\rangle$ is isomorphic to $\langle ad,be,cf\rangle$. From the multiplicativity of the matrix multiplication tensors it is easy to derive that $\omega$ is characterized by the asymptotic rank of $\langle 2,2,2\rangle$ as follows.

Proposition 1** (see, e.g., [Blä13, Theorem 5.2]).**

$\omega=\log_{2}\operatorname{\underaccent{\wtilde}{R}}(\langle 2,2,2\rangle)$.

The difficulty of determining the asymptotic rank of $\langle 2,2,2\rangle$ is to be contrasted with the situation for the asymptotic subrank; to put it in Strassen’s words: Unlike the cynic, who according to Oscar Wilde knows the price of everything and the value of nothing, we can determine the asymptotic value of $\langle h,h,h\rangle$ precisely.

Proposition 2** ([Str88]).**

$2=\log_{2}\operatorname{\underaccent{\wtilde}{Q}}(\langle 2,2,2\rangle)$. More precisely, for any $h\in\mathbb{N}$ it holds that

[TABLE]

For completeness of the paper we give a proof of Proposition 2. The proof we provide here is based on Salem–Spencer sets and is slightly different from the proof in [Str88]. However, the two proofs become very similar when the construction of Salem–Spencer sets is unrolled.

Proof.

Since each flattening rank of the diagonal tensor $I_{n}$ equals $n$ and since the flattening rank of $\langle h,h,h\rangle$ equals $h^{2}$ we find by multiplicativity and monotonicity of the flattening rank that $\operatorname{\underaccent{\wtilde}{Q}}(\langle h,h,h\rangle)\leq h^{2}$. We must prove the matching lower bound. Let $p$ be a large prime. There is a subset $A\subseteq\mathbb{Z}/p\mathbb{Z}$ (a Salem–Spencer set in $\mathbb{Z}/p\mathbb{Z}$) that is free of nontrivial three-term arithmetic progressions and that has size $\mathinner{\lvert A\rvert}=p^{1-o(1)}$ [SS42, Beh46]. The support of the tensor $\langle p,p,p\rangle$ is the set of triples

[TABLE]

We think of $E$ as a 3-partite 3-uniform hypergraph $E\subseteq V_{1}\times V_{2}\times V_{3}$ with vertex sets $V_{\ell}=[p]\times[p]$. Consider the sub-hypergraph $F\subseteq E$ induced by the vertex sets

[TABLE]

That is, $F=E\cap(W_{1}\times W_{2}\times W_{3})$. Then

[TABLE]

Indeed, a priori, $((i,j),(j,k),(k,i))\in F$ if and only if

[TABLE]

for some $x\in[p]$ and $a,b,c\in A$. Then $a,(a+b)/2,b$ is a three-term arithmetic progression in $A$, and thus $a=b$ and the claim follows. We see that the set $F$ has $p\mathinner{\lvert A\rvert}=p^{2-o(1)}$ elements. We also see that for any triple $((i,j),(j,k),(k,i))$ in $F$, the first vertex $(i,j)$ determines the second vertex $(j,k)$ and the third vertex $(k,i)$, and similarly the second vertex determines the first and third vertex and the third vertex determines the first and second vertex. In other words, the edges in $F$ are disjoint in all coordinates, that is, they form a matching. Restrict the tensor $\langle p,p,p\rangle\in\mathbb{F}^{V_{1}\times V_{2}\times V_{3}}$ to $\mathbb{F}^{W_{1}\times W_{2}\times W_{3}}$ by restricting the coordinates in each direction from $V_{\ell}$ to $W_{\ell}$. The support of the resulting tensor is the matching $F$. After a suitable permutation of each $W_{\ell}$, this tensor becomes a diagonal tensor of size $|F|=p^{2-o(1)}$. We conclude that for every prime $p$ we have $\operatorname{Q}(\langle p,p,p\rangle)\geq p^{2-o(1)}$. By a simple tensor product argument this implies the claim for all $h\in\mathbb{N}$. Namely, for every $h\in\mathbb{N}$ we take a prime $p$ such that $h^{n}$ is not much bigger than an integer power of $p$, that is, $h^{n}\geq p^{m}\geq h^{n}/p$. Then $\langle h^{n},h^{n},h^{n}\rangle\geq\langle p^{m},p^{m},p^{m}\rangle$ and we find that $\operatorname{Q}(\langle h^{n},h^{n},h^{n}\rangle)\geq h^{2n-o(n)}$. ∎

Remark 3**.**

We remarked before that Strassen’s proof of Proposition 2 in [Str88] takes a slightly different route. Namely, it makes use of an approximative relaxation of subrank called border subrank, which is denoted by $\operatorname{\underline{Q}}$. He shows that $\operatorname{\underline{Q}}(\langle m,m,m\rangle)\geq\lceil\tfrac{3}{4}m^{2}\rceil$. (This lower bound, and all Strassen’s lower bounds for $\operatorname{\underline{Q}}(\langle a,b,c\rangle)$, are in fact tight [KMZ20].) Strassen then relates the border subrank to the asymptotic subrank via a simple polynomial interpolation argument to get that $\operatorname{\underaccent{\wtilde}{Q}}(\langle m,m,m\rangle)=m^{2}$.

2.2. Relative exponent

For a clean exposition of our barrier we will use the notion of relative exponent, which we will define in this section. This notion is inspired by the notion of rate from information theory and alternatively can be seen as a versatile version of the notion of the asymptotic preorder for tensors of Strassen222The asymptotic preorder $\gtrsim$ is defined as follows: $s\gtrsim t$ if $s^{\otimes n+o(n)}\geq t^{\otimes n}$ when $n\to\infty$.. In the context of tensors, the relative exponent previously appeared in [YGD14],[VC15] and [CVZ19].

To avoid technicalities, we will from now on, without further mentioning, only consider tensors that are not of tensor rank one or zero. This assumption is explicitly used in the proof of Proposition 5(i) and its applications.

Definition 4**.**

For two tensors $t\in\mathbb{F}^{n_{1}\times n_{2}\times n_{3}}$ and $s\in\mathbb{F}^{m_{1}\times m_{2}\times m_{3}}$ we define the relative exponent from $t$ to $s$ as

[TABLE]

The limit is a supremum by Fekete’s lemma. Let us briefly relate the relative exponent to the basic notions and results stated earlier. The reader verifies directly that the identities

[TABLE]

hold by approximating integers by powers of 2, or see [CVZ19, Proposition 1.1.16] for a proof. From the fact that $\omega\coloneqq\log_{2}\operatorname{\underaccent{\wtilde}{R}}(\langle 2,2,2\rangle)$ (Proposition 1) it follows that

[TABLE]

We know from (7) that $\operatorname{\underaccent{\wtilde}{Q}}(\langle 2,2,2\rangle)=4$ and so

[TABLE]

The relative exponent has the following two basic properties.

Proposition 5**.**

Let $s$, $t$ and $u$ be tensors.

1. (i)

$\operatorname{\omega}(t,t)=1$. 2. (ii)

$\operatorname{\omega}(s,t)\operatorname{\omega}(t,u)\geq\operatorname{\omega}(s,u)$*  (triangle inequality).*

Proof.

(i) Clearly $\omega(t,t)\leq 1$ by reflexivity of the restriction preorder $\geq$. Since $t$ is not of rank 1, we can flatten $t$ into a matrix in one of the three directions, so that the matrix rank is some number $r>1$. It follows from multiplicativity of matrix rank, that if $t^{\otimes m}\geq t^{\otimes n}$, then $r^{m}\geq r^{n}$ and so $m\geq n$. This implies the claim. (ii) follows from the transitivity of the restriction preorder $\geq$. ∎

2.3. Irreversibility.

Our barrier framework relies crucially on the irreversibility of a tensor, a new notion that we define now.

Definition 6**.**

We define the irreversibility of a tensor $t$ as the product of the relative exponent from $\langle 2\rangle$ to $t$ and the relative exponent from $t$ to $\langle 2\rangle$, i.e.

[TABLE]

Thus $\operatorname{\mathbf{i}}(t)$ measures the extent to which the asymptotic conversion from $\langle 2\rangle$ to $t$ is irreversible, explaining the name. Equivalently, the irreversibility is the ratio of the logarithms of the asymptotic rank and the asymptotic subrank, i.e.

[TABLE]

Proposition 7**.**

For any tensor $t$ it holds that

[TABLE]

Proof.

From the basic properties of the relative exponent (Proposition 5) it follows directly that the inequality $\operatorname{\mathbf{i}}(t)=\operatorname{\omega}(\langle 2\rangle,t)\operatorname{\omega}(t,\langle 2\rangle)\geq\operatorname{\omega}(\langle 2\rangle,\langle 2\rangle)=1$ holds. ∎

Definition 8**.**

We call a tensor $t$ reversible if $\operatorname{\mathbf{i}}(t)=1$ and irreversible otherwise (in which case it holds that $\operatorname{\mathbf{i}}(t)>1$ by Proposition 7).

For example, for any $n\in\mathbb{N}$ the diagonal tensor $\langle n\rangle=\sum_{i=1}^{n}e_{i,i,i}$ is reversible. In fact, every reversible tensor $t$, that we know of, is equivalent to $\langle n\rangle$ for some $n$, in the sense that $\langle n\rangle\leq t\leq\langle n\rangle$. In other words, we do not know whether every reversible tensor $t$ is equivalent to $\langle n\rangle$ for some $n$.

This question is closely related to whether the matrix multiplication exponent $\omega$ equals 2. For the matrix multiplication tensor $\langle 2,2,2\rangle$ we have that $2\operatorname{\mathbf{i}}(\langle 2,2,2\rangle)=\omega$ (using (13)). Thus $\omega=2$ if and only if $\langle 2,2,2\rangle$ is reversible. (In fact, for any $n\in\mathbb{N}$ it holds that $\omega=2$ if and only if $\langle n,n,n\rangle$ is reversible.) As we will see in Section 3, this is ultimately the source of our barrier.

Irreversible tensors do in fact exist. For example, the tensor $W=e_{0,0,1}+e_{0,1,0}+e_{1,0,0}$ is irreversible. Namely, it is known that $\log_{2}\operatorname{\underaccent{\wtilde}{R}}(W)=1$ and that $\log_{2}\operatorname{\underaccent{\wtilde}{Q}}(W)=h(1/3)=0.918..$ [Str91, Theorem 6.7], so $\operatorname{\mathbf{i}}(W)=1.088..>1$. In Section 5 we will compute lower bounds on the irreversibility of the small and big Coppersmith–Winograd tensors (which play a crucial role in the best upper bounds on $\omega$).

2.4. Monomial relative exponent and monomial irreversibility

The following restricted version of relative exponent and irreversibility will be relevant. A generalized sub-permutation matrix is a matrix for which every row and every column has at most one nonzero coefficient. For two tensors $t\in\mathbb{F}^{n_{1}\times n_{2}\times n_{3}}$ and $s\in\mathbb{F}^{m_{1}\times m_{2}\times m_{3}}$ we write $t\geq_{\mathrm{M}}s$ and say $t$ monomially restricts to $s$ if there are linear maps $A_{i}\mathrel{\mathop{\mathchar 58\relax}}\mathbb{F}^{n_{i}}\to\mathbb{F}^{m_{i}}$, the corresponding matrices of which are generalized sub-permutation matrices in the standard basis, such that $(A_{1},A_{2},A_{3})\cdot t=s$ [Str87, Section 6]. Thus $s$ is a monomial restriction of $t$ if we can obtain $s$ from $t$ by rescaling (also to zero) and permuting the slices of $t$. Monomial restriction plays a central role in the existing upper bounds on $\omega$.

Replacing the preorder $\geq$ by $\geq_{\mathrm{M}}$ in Section 2 gives the notions of monomial subrank $\operatorname{Q}_{\mathrm{M}}$, monomial asymptotic subrank $\underaccent{\wtilde}{\mathrm{Q}}_{\mathrm{M}}$ and monomial relative exponent $\operatorname{\omega_{\mathrm{M}}}$. (For simplicity we will use monomial restriction here, but our results will also hold with $\geq_{\mathrm{M}}$ replaced by monomial degeneration $\unrhd_{\mathrm{M}}$ defined in [Str87, Section 6]. Monomial degeneration is the approximative version of monomial restriction. It is reflexive and transitive like monomial restriction.) Note that the notions $\operatorname{Q}_{\mathrm{M}}$ and $\underaccent{\wtilde}{\mathrm{Q}}_{\mathrm{M}}$ only depend on the support of the tensor, and not on the particular values of the nonzero coefficients. We define the monomial irreversibility $\operatorname{\mathbf{i}_{\mathrm{M}}}(t)$ of $t$ as the product of the (normal) relative exponent from $\langle 2\rangle$ to $t$ and the monomial relative exponent from $t$ to $\langle 2\rangle$,

[TABLE]

Equivalently, we have

[TABLE]

We stress that the monomial irreversibility uses the monomial asymptotic subrank, but the regular asymptotic rank. This is in fact what is used in practice. We note that monomial (asymptotic) rank is a useless concept, since any monomial restriction of a unit tensor is again a unit tensor. In other words, using the monomial asymptotic rank is too restrictive and gives bad algorithms from the start. In particular, monomial irreversibility depends on the tensor and not only on its support.

Proposition 9**.**

Let $s$, $t$ and $u$ be tensors.

1. (i)

$\operatorname{\omega_{\mathrm{M}}}(t,t)=1$. 2. (ii)

$\operatorname{\omega_{\mathrm{M}}}(s,t)\operatorname{\omega_{\mathrm{M}}}(t,u)\geq\operatorname{\omega_{\mathrm{M}}}(s,u)$*  (triangle inequality).* 3. (iii)

$\operatorname{\omega_{\mathrm{M}}}(s,t)\geq\operatorname{\omega}(s,t)$. 4. (iv)

$\operatorname{\mathbf{i}_{\mathrm{M}}}(t)\geq\operatorname{\mathbf{i}}(t)$.

Proof.

The first two statements follow from reflexivity and transitivity of the monomial restriction preorder $\geq_{\mathrm{M}}$, just like in the proof of Proposition 5. The other two statements follow from the fact that $s\geq_{\mathrm{M}}t$ implies $s\geq t$. ∎

Definition 10**.**

We call a tensor $t$ monomially reversible if $\operatorname{\mathbf{i}_{\mathrm{M}}}(t)=1$ and monomially irreversible otherwise (in which case it holds that $\operatorname{\mathbf{i}_{\mathrm{M}}}(t)>1$ by Proposition 9 (iv) and Proposition 7).

There exist tensors that are reversible and monomially irreversible. We give an example.

Example 11**.**

For any finite group $G$, we define the tensor $\langle G\rangle_{\mathbb{F}}\in\mathbb{F}^{G\times G\times G}$ by

[TABLE]

The tensor $\langle G\rangle_{\mathbb{F}}$ is often referred to as the structure tensor of the group algebra $\mathbb{F}[G]$. The group algebra $\mathbb{F}[G]$ is the vector space $\mathbb{F}^{G}$ that has a bilinear multiplication operation defined on it by setting $e_{g}\cdot e_{h}=e_{gh}$ for the basis elements $\{e_{g}\mathrel{\mathop{\mathchar 58\relax}}g\in G\}$ and extending bilinearly to $\mathbb{F}[G]$. Thus $\langle G\rangle_{\mathbb{F}}$ precisely encodes this multiplication. We stress that in the notation $\langle G\rangle_{\mathbb{F}}$ the parameter $G$ determines the support and the parameter $\mathbb{F}$ denotes over which field we consider the tensor. Even though all coefficients of $\langle G\rangle_{\mathbb{F}}$ are 0 or 1, the choice of base field $\mathbb{F}$ determines what linear maps are allowed in the restriction preorder $\geq$ and thus the notion of equivalence and the notion of isomorphism. Crucially, for the monomial restriction preorder $\geq_{\mathrm{M}}$ applied to tensors with coefficients that are only 0 or 1, the choice of base field does not make a difference.

We consider the structure tensor

[TABLE]

First of all, $\langle\mathbb{Z}/3\mathbb{Z}\rangle_{\mathbb{C}}$ is isomorphic to the diagonal tensor $\langle 3\rangle$. In general, for any finite abelian group $A$, the structure tensor $\langle A\rangle_{\mathbb{C}}$ is isomorphic to the diagonal tensor $\langle|A|\rangle$, using the Fourier transform on $A$ to diagonalize the multiplication (see, e.g., [BCC*+*17a] or [CVZ18] for a discussion of this). Therefore, $\operatorname{\underaccent{\wtilde}{R}}(\langle\mathbb{Z}/3\mathbb{Z}\rangle_{\mathbb{C}})=3$ and $\operatorname{\underaccent{\wtilde}{Q}}(\langle\mathbb{Z}/3\mathbb{Z}\rangle_{\mathbb{C}})=3$ and hence $\langle\mathbb{Z}/3\mathbb{Z}\rangle_{\mathbb{C}}$ is reversible.

In fact, for general $\mathbb{F}$, we have $\operatorname{\underaccent{\wtilde}{R}}(\langle\mathbb{Z}/3\mathbb{Z}\rangle_{\mathbb{F}})=3$ because there is a monomial degeneration from the restricted polynomial multiplication tensor to $\langle\mathbb{Z}/3\mathbb{Z}\rangle_{\mathbb{F}}$, see [CVZ18, Theorem 4.16]. On the other hand, it is known that $\underaccent{\wtilde}{\mathrm{Q}}_{\mathrm{M}}(\langle\mathbb{Z}/3\mathbb{Z}\rangle_{\mathbb{F}})=2.755..$. This is proven in [EG17, Tao16], see also [CVZ18] for the connection to [Str91]. It follows for any field $\mathbb{F}$ that $\operatorname{\mathbf{i}_{\mathrm{M}}}(\langle\mathbb{Z}/3\mathbb{Z}\rangle_{\mathbb{F}})=1.08..$ and that $\langle\mathbb{Z}/3\mathbb{Z}\rangle_{\mathbb{F}}$ is monomially irreversible.

Regarding matrix multiplication, Strassen’s construction for (13) (see also our proof of Proposition 2) in fact shows that

[TABLE]

2.5. Upper bounds on irreversibility

We finish this section by discussing the possible values that the irreversibility can take, as a first step towards a systematic understanding of where we may find reversible tensors or almost reversible tensors. We will see that, surprisingly at first sight, the matrix multiplication exponent $\omega$ provides bounds on the irreversibility of arbitrary tensors.

First of all, if $\operatorname{\underaccent{\wtilde}{Q}}(t)\leq 1$, then $\operatorname{\mathbf{i}}(t)=\infty$. Thus, a priori, the irreversibility of a tensor could be any number in $[1,\infty]$. Recall that we defined the flattenings of any tensor $t\in\mathbb{F}^{n_{1}\times n_{2}\times n_{3}}$ as the linear maps $t_{1}\mathrel{\mathop{\mathchar 58\relax}}\mathbb{F}^{n_{1}}\to\mathbb{F}^{n_{2}\times n_{3}}$, $t_{2}\mathrel{\mathop{\mathchar 58\relax}}\mathbb{F}^{n_{2}}\to\mathbb{F}^{n_{1}\times n_{3}}$ and $t_{3}\mathrel{\mathop{\mathchar 58\relax}}\mathbb{F}^{n_{3}}\to\mathbb{F}^{n_{1}\times n_{2}}$ that are naturally associated to $t$. For any tensor $t$ that has a flattening with rank (as a linear map) at most 1 it holds that $\operatorname{\underaccent{\wtilde}{Q}}(t)\leq 1$. For example, $\operatorname{\underaccent{\wtilde}{Q}}(\langle n,1,1\rangle)=1$. If none of the flattening ranks is at most 1, then the asymptotic subrank is at least some absolute constant $c>1$, as follows from [Str88, Lemma 3.7]. Thus in that case the irreversibility is finite. On the other hand, for any $t\in\mathbb{F}^{n\times n\times n}$ the upper bound $\operatorname{\underaccent{\wtilde}{R}}(t)\leq\operatorname{R}(t)\leq n^{2}$ holds. We conclude that, for $t\in\mathbb{F}^{n\times n\times n}$, if any flattening rank is at most 1, then $\operatorname{\mathbf{i}}(t)=\infty$, and otherwise

[TABLE]

for some absolute constant $d$. This gives us an idea of the order of magnitude of the irreversibility. We will refine this upper bound in the rest of this discussion.

As the first step towards gaining more control on the possible values of the irreversibility, we note that for asymptotic rank we have the following general upper bound.

Proposition 12** ([Str88, Proposition 3.6]).**

For $t\in\mathbb{F}^{n\times n\times n}$ it holds that $\operatorname{\underaccent{\wtilde}{R}}(t)\leq n^{2\omega/3}$.

Proof.

We give a sketch of the argument. For any tensor $t\in\mathbb{F}^{n\times n\times n}$ it holds that $t\leq\langle n,n,1\rangle$ and $t\leq\langle n,1,n\rangle$ and $t\leq\langle 1,n,n\rangle$. By multiplying these inequalities, it follows that $t^{\otimes 3}\leq\langle n^{2},n^{2},n^{2}\rangle$. It is not hard to see that therefore $\operatorname{\underaccent{\wtilde}{R}}(t)\leq\operatorname{\underaccent{\wtilde}{R}}(\langle n,n,n\rangle)^{2/3}=n^{2\omega/3}$. ∎

It is possible that $\omega=2$, in which case it follows from Proposition 12 that $\operatorname{\underaccent{\wtilde}{R}}(t)\leq n^{4/3}$ for every $t\in\mathbb{F}^{n\times n\times n}$. More generally, it is possible that for every tensor $t\in\mathbb{F}^{n\times n\times n}$ we have the highly non-trivial upper bound $\operatorname{\underaccent{\wtilde}{R}}(t)\leq n$, see [BCS97, Problem 15.5]. Strassen conjectured this to be true for a subset of all tensors called tight tensors [Str94, Conjecture 5.3].

Proposition 12 gives us some control over the possible values of the irreversibility. As the next step in that direction we must understand how small the asymptotic subrank can be. First, we give an example of a tensor for which the asymptotic subrank and the asymptotic rank are relatively far apart.

Example 13**.**

Let $t\in\mathbb{F}^{n\times n\times n}$ be the tensor $t=e_{1,n,1}+e_{n,1,1}+e_{2,n,2}+e_{n,2,2}+\cdots+e_{n,n,n}$. Then for $n\geq 5$ it is known [Str91, Equation (6.19)] that $\operatorname{\underaccent{\wtilde}{Q}}(t)=2\sqrt{n-1}$ while $\operatorname{\underaccent{\wtilde}{R}}(t)=n$ and so we have that $\operatorname{\mathbf{i}}(t)$ goes to 2 (from below) when $n$ goes to infinity.

For the rest of the discussion we require our tensors to satisfy a property called balanced. This property was introduced in [Str88, page 121]. Not all tensors are balanced, but in any tensor space $\mathbb{F}^{n\times n\times n}$ over an algebraically closed field $\mathbb{F}$, being balanced is a generic condition in the sense of algebraic geometry. In particular, almost all elements in $\mathbb{F}^{n\times n\times n}$ are balanced. We define a tensor $t\in\mathbb{F}^{n_{1}\times n_{2}\times n_{3}}$ to be balanced if the three flattenings that we defined before are full-rank as linear maps and for each $i\in[3]$ there is an element $v\in\mathbb{F}^{n}$ such that the element $t_{i}(v)$ has full rank (as a matrix). Balanced tensors are called 1-generic tensors in [LM17]. An example of a non-balanced tensor is the tensor in Example 13. An example of a balanced tensor is the matrix multiplication tensor $\langle n,n,n\rangle$.

Proposition 14**.**

Let $t\in\mathbb{F}^{n\times n\times n}$ be balanced. Then

[TABLE]

Proof.

The claim follows from the following two ingredients. The first ingredient is the general upper bound $\operatorname{\underaccent{\wtilde}{R}}(t)\leq n^{2\omega/3}$ from Proposition 12. The second ingredient is the lower bound $\operatorname{\underaccent{\wtilde}{Q}}(t)\geq n^{2/3}$ [Str88, Proposition 3.6] for balanced tensors. We give a sketch of the argument. From the balancedness assumption it follows that $\langle n,1,1\rangle\leq t$ and $\langle 1,n,1\rangle\leq n$ and $\langle 1,1,n\rangle\leq t$. By multiplying these inequalities we get $\langle n,n,n\rangle\leq t^{\otimes 3}$. Therefore, using Proposition 2, we have $n^{2}\leq\operatorname{\underaccent{\wtilde}{Q}}(\langle n,n,n\rangle)\leq\operatorname{\underaccent{\wtilde}{Q}}(t)^{3}$. ∎

Again, it is possible that $\omega=2$, in which case it follows from Proposition 14 that $1\leq\operatorname{\mathbf{i}}(t)\leq 2$ for balanced $t$. More generally, it is possible that for every tensor $t\in\mathbb{F}^{n\times n\times n}$ we have the highly non-trivial upper bound $\operatorname{\underaccent{\wtilde}{R}}(t)\leq n$. In that case we have the following bounds.

Proposition 15**.**

Let $t\in\mathbb{F}^{n\times n\times n}$ be balanced and satisfy $\operatorname{\underaccent{\wtilde}{R}}(t)\leq n$. Then

[TABLE]

Proof.

The claim follows directly from combining $\operatorname{\underaccent{\wtilde}{Q}}(t)\geq n^{2/3}$, which follows from balancedness as we saw in the proof of Proposition 14, and the assumption $\operatorname{\underaccent{\wtilde}{R}}(t)\leq n$. ∎

Example 16**.**

The upper bound in Proposition 15 is tight. Namely, let $t=\mathrm{cw}_{q}$ be the small Coppersmith–Winograd tensor with parameter $q$. We will see from Theorem 25 that $\operatorname{\mathbf{i}}(t)\rightarrow 1.5$ when $q$ goes to infinity.

3. Irreversibility implies barriers

With the new notion of irreversibility available, we present a barrier for approaches to upper bound $\omega$ via an intermediate tensor $t$. As we have discussed before, all recent successful upper bounds on $\omega$ have been obtained with constructions via an intermediate tensor. The results in this section tell us which tensors not to use as intermediate tensors and what necessary quality a good intermediate tensor has.

3.1. The irreversibility barrier

For any tensor $t$ the inequality

[TABLE]

holds by the triangle inequality. Any such approach to upper bound $\omega$ respects the following barrier in terms of the irreversibility $\operatorname{\mathbf{i}}(t)$ of $t$.

Theorem 17**.**

For any tensor $t$ it holds that

[TABLE]

Proof.

By the triangle inequality (Proposition 5),

[TABLE]

Therefore, using the fact $\omega(\langle 2,2,2\rangle,\langle 2\rangle)=\tfrac{1}{2}$ from (13), we have

[TABLE]

This proves the claim. ∎

Theorem 17, in particular, implies that if $\operatorname{\mathbf{i}}(t)>1$, then $\operatorname{\omega}(\langle 2\rangle,t)\operatorname{\omega}(t,\langle 2,2,2\rangle)>2$. In other words, we cannot prove $\omega=2$ via an irreversible intermediate tensor. However, it is possible that there exists a sequence of irreversible intermediate tensors with irreversibility converging to 1 that can be used to prove $\omega=2$.

To conclude, the barrier just introduced describes what quality makes a tensor a good intermediate tensor, or put differently what intermediate tensors definitely not to use when we want to prove good upper bounds on $\omega$. Of course the barrier does not tell us explicitly which intermediate tensor to pick, but it provides a strong heuristic of what to look for, namely low asymptotic rank and high asymptotic subrank.

3.2. Better barriers when the method has more structure

The barrier discussed in Section 3.1 does not tell the full story, and we will now discuss the natural and more subtle continuation. Namely, not only will we lose the game when the intermediate tensor is irreversible, we lose even more when we use this intermediate tensor in a catalytic fashion, meaning that our algorithm is obtained from transforming a diagonal tensor to the tensor product of a diagonal tensor and a matrix multiplication tensor (via the intermediate tensor). We will explain this more, but for now it is important to know that this strategy is commonly used in the literature (with great success). However, as we will see in this section, such a catalytic approach boosts the previous barrier even more. Thus this section gives rise to a more precise heuristic which says that, while looking for intermediate tensors with small irreversibility, we may at the same time want to think about intermediate tensors that require less use of catalysis.

Catalysis in the general context of tensors is the phenomenon that for tensors $s$, $t$ and $u$, the inequality $s\geq t$ may be false, while the inequality $s\otimes u\geq t\otimes u$ may be true. The latter inequality is called catalytic with the tensor $u$ acting as a catalyst.

Catalysis is widely used in matrix multiplication algorithms albeit not under this explicit terminology. We will now discuss more quantitatively what we mean when we say that an approach to upper bound $\omega$ uses catalysis. Without loss of generality we may impose that the final step of any construction of a matrix multiplication algorithm is an application of the Schönhage $\tau$-theorem. The Schönhage $\tau$-theorem (Strassen’s general version [Str88]) says that

[TABLE]

In particular, for $a_{i}=b_{i}=c_{i}=a$, it holds that

[TABLE]

(And this inequality is in fact tight, although we will not go into that here.) We are using here that the direct sum of $q$ copies of a tensor $t$ is isomorphic to the tensor product of the diagonal tensor $\langle q\rangle$ and $t$. Equivalently, in the language of rates, for any $\alpha,\beta\in\mathbb{N}$ it holds that

[TABLE]

that is

[TABLE]

(Here $\alpha$ corresponds to $\log_{2}q$ and $\beta$ corresponds to $\log_{2}a$. For simplicity and concreteness we will consider only integer $\alpha$ and $\beta$.) Thus for any tensor $t$ and for any $\alpha,\beta\in\mathbb{N}$ it holds that

[TABLE]

This inequality (29) is a method for upper bounding $\omega$ and we say that it is catalytic when $\alpha>0$, the catalyst being the tensor $\langle 2\rangle^{\alpha}$. In fact, upper bounds coming from the approach in the Coppersmith–Winograd paper and its follow-ups take the form of this inequality for specific $t$, $\alpha$ and $\beta$. The following barrier in terms of $\alpha,\beta$ and the irreversibility $\operatorname{\mathbf{i}}(t)$ of $t$ for any method of the form (29) says that catalysis boosts the irreversibility barrier.

Theorem 18**.**

For any tensor $t$ and $\alpha,\beta\in\mathbb{N}$ it holds that

[TABLE]

Proof.

By the triangle inequality,

[TABLE]

Therefore,

[TABLE]

We can bound this by

[TABLE]

since for any tensors $s,t,u$ the inequality $\operatorname{\omega}(s\otimes t,u)^{-1}\geq\operatorname{\omega}(s,u)^{-1}+\operatorname{\omega}(t,u)^{-1}$ holds. (The inequality in (33) is actually known to be an equality, since the asymptotic subrank is known to be additive on any collection of tensors of the form $\{\oplus_{i}t^{\otimes{n_{i}}}\mathrel{\mathop{\mathchar 58\relax}}n_{i}\in\mathbb{N}\}$ for any fixed tensor $t$. This for example follows from the asymptotic subrank being characterized by the asymptotic spectrum of tensors that we will discuss in the next section.) Combining (32) and (33), subtracting $\alpha$, dividing by $\beta$ and using that $\operatorname{\mathbf{i}}(t)-1\geq 0$ (Proposition 7) gives the barrier

[TABLE]

This proves the claim. ∎

As a corollary of the above theorem, we present a barrier on any approach of the following form. The Schönhage $\tau$-theorem implies that for any $a,b,c\in\mathbb{N}_{\geq 1}$ and any tensor $t$ it holds that

[TABLE]

Again, we think of this inequality as a method for upper bounding $\omega$. To prove a barrier for this method we define a cyclic symmetrization of $t$. For any tensor $t=(t_{ijk})\in\mathbb{F}^{n\times n\times n}$ we define the cyclic permutations $(1,2,3)\cdot t$ and $(1,2,3)^{2}\cdot t$ by cyclically permuting the indices $i$, $j$ and $k$. We define the cyclic symmetrization $\operatorname{cyc}(t)$ by $\operatorname{cyc}(t)\coloneqq t\otimes((1,2,3)\cdot t)\otimes((1,2,3)^{2}\cdot t)\in\mathbb{F}^{n^{3}\times n^{3}\times n^{3}}$. For example, $\operatorname{cyc}(\langle a,b,c\rangle)=\langle a,b,c\rangle\otimes\langle c,a,b\rangle\otimes\langle b,c,a\rangle=\langle abc,abc,abc\rangle$. We prove the following barrier in terms of $a,b,c$, $\alpha$ and the irreversibility of $\operatorname{cyc}(t)$.

Corollary 19**.**

For any tensor $t$ and $\alpha\in\mathbb{N}$ and $a,b,c\in\mathbb{N}_{\geq 1}$ it holds that

[TABLE]

One verifies that $\operatorname{\mathbf{i}}(t)\geq\operatorname{\mathbf{i}}(\operatorname{cyc}(t))$. If $t$ is cyclically symmetric, then $\operatorname{cyc}(t)=t^{\otimes 3}$ and we have the equality $\operatorname{\mathbf{i}}(t)=\operatorname{\mathbf{i}}(\operatorname{cyc}(t))$.

Proof.

We first prove that

[TABLE]

Suppose that $\langle 2\rangle^{m}\geq t^{n}$. Then also $\langle 2\rangle^{m}\geq((1,2,3)\cdot t)^{n}$ and $\langle 2\rangle^{m}\geq((1,2,3)^{2}\cdot t)^{n}$. Multiplying these inequalities gives $\langle 2\rangle^{3m}\geq\operatorname{cyc}(t)^{n}$. We conclude that $m/n\geq\omega(\langle 2\rangle,\operatorname{cyc}(t)^{1/3})$. By a similar argument we find that

[TABLE]

Note that we are using real powers of tensors here inside the relative exponent $\omega(\cdot,\cdot)$. This is justified by taking powers of the relevant tensors and taking a limit. Using both inequalities and then applying Theorem 18 gives

[TABLE]

This proves the statement of the theorem. ∎

3.3. Better barriers through monomial irreversibility

Finally, we impose as an extra constraint that the transformation from the intermediate tensor $t$ to the matrix multiplication tensor happens via monomial restriction (Section 2.4), that is, we consider the approach

[TABLE]

and the more structured approaches

[TABLE]

and

[TABLE]

The proofs in the previous sections can be directly adapted to prove:

Theorem 20**.**

For any tensor $t$ it holds that

[TABLE]

Theorem 21**.**

For any tensor $t$ and $\alpha,\beta\in\mathbb{N}$ it holds that

[TABLE]

Corollary 22**.**

For any tensor $t$ and $\alpha\in\mathbb{N}$ and $a,b,c\in\mathbb{N}_{\geq 1}$ it holds that

[TABLE]

4. Methods for lower bounding irreversibility

We have seen how lower bounds on irreversibility imply barriers. By definition of irreversibility, such lower bounds come from lower bounds on asymptotic rank and upper bounds on asymptotic subrank. In this section we discuss lower bounding irreversibility from four points of view: the asymptotic spectrum of tensors, the support functionals, the quantum functionals and the asymptotic slice rank. We will use the support functionals to compute explicit barriers in Section 5; the rest of this section serves as a survey and to provide comparison.

4.1. The asymptotic spectrum of tensors

We begin with an abstract and clean approach to characterizing irreversibility, and will move to practical bounds in the following sections. Let $S$ be a family of tensors that is closed under $\otimes$ and $\oplus$ and that contains $\langle 1\rangle$. Strassen [Str88] introduced the asymptotic spectrum of tensors $\Delta(S)$ as the set of all maps $F\mathrel{\mathop{\mathchar 58\relax}}S\to\mathbb{R}_{\geq 0}$ that satisfy for any tensors $s$ and $t$ in $S$ that

• •

$s\leq t\Rightarrow F(s)\leq F(t)$

• •

$F(s\otimes t)=F(s)F(t)$

• •

$F(s\oplus t)=F(s)+F(t)$

• •

$F(\langle 1\rangle)=1$.

Strassen proved in [Str88] that for any tensor $t\in S$ it holds that $\operatorname{\underaccent{\wtilde}{Q}}(t)=\min_{F\in\Delta(S)}F(t)$ and $\operatorname{\underaccent{\wtilde}{R}}(t)=\max_{F\in\Delta(S)}F(t)$. From this we directly obtain the following concise characterization of irreversibility in terms of the asymptotic spectrum of tensors $\Delta(S)$.

Proposition 23**.**

Let $t$ be a tensor in $S$. Then

[TABLE]

This characterization is clean but does not lead to a practical way of computing, or even lower bounding, the irreversibility $\operatorname{\mathbf{i}}(t)$ for a general tensor $t$. This is because our knowledge of $\Delta(S)$ is very limited for any general family of tensors $S$. Namely, a priori we only know that for every $i\in[3]$ the function $t\mapsto\operatorname{R}(t_{i})$ is in $\Delta(S)$, where $t_{i}$ is the flattening as described in Section 2.5. Thus we have the inequalities $\operatorname{\underaccent{\wtilde}{Q}}(t)\leq\min_{i}\operatorname{R}(t_{i})$ and $\operatorname{\underaccent{\wtilde}{R}}(t)\geq\max_{i}\operatorname{R}(t_{i})$. In fact, $\max_{i}\operatorname{R}(t_{i})$ is the best lower bound on $\operatorname{\underaccent{\wtilde}{R}}(t)$ that we know of. We will keep using this lower bound on the asymptotic rank in the coming, more practical, sections. There are, however, more powerful tools than the flattening ranks $\operatorname{R}(t_{i})$ to upper bound $\operatorname{\underaccent{\wtilde}{Q}}(t)$, which we will discuss now.

4.2. The support functionals

The support functionals of Strassen provide an extremely powerful method for upper bounding the asymptotic subrank of tensors (and hence lower bounding irreversibility). The definition of the support functionals is technical at first sight, but as a reward these compact objects provide us with the best upper bounds on the asymptotic subrank. Before we define the support functionals, we need to define some notions for tensors and for probability distributions. For tensors $s,t\in\mathbb{F}^{n_{1}\times n_{2}\times n_{3}}$ we write $s\cong t$ and say that $s$ and $t$ are isomorphic if there are invertible linear maps $A_{i}$ such that $(A_{1},A_{2},A_{3})\cdot s=t$. Recall that we denote by

[TABLE]

the support of $s$. For any probability distribution $Q$ on $[n]$ let $H(Q)=\sum_{i\in[n]}Q(i)\log_{2}(1/Q(i))$ denote the Shannon entropy of $Q$. For any probability distribution $P$ on $[n_{1}]\times[n_{2}]\times[n_{3}]$ let $P_{i}$ be the marginal distribution on $[n_{i}]$. That is, the marginal distribution $P_{1}$ is defined by $P_{1}(i_{1})\coloneqq\sum_{i_{2}\in[n_{2}]}\sum_{i_{3}\in[n_{3}]}P(i_{1},i_{2},i_{3})$ and the other marginal distributions are defined similarly. Let $\mathcal{P}(\operatorname{supp}(s))$ denote the set of all probability distributions on $\operatorname{supp}(s)$. (Thus for any element $P\in\mathcal{P}(\operatorname{supp}(s))$ we can talk about its marginal distributions $P_{i}$ and their entropies $H(P_{i})$.) For any tensor $t$ and any probability vector $\theta\in\mathbb{R}^{3}$ Strassen defined the support functional $\zeta^{\theta}$ by setting

[TABLE]

where

[TABLE]

(We note that we believe that the name support functional derives from the general concept of support functions in convex analysis, rather than the support of tensors.) The minimization over all $s$ isomorphic to $t$ appearing in the definition of $\rho^{\theta}(t)$ is generally not well understood, but, fortunately, for the sake of upper bounding $\zeta^{\theta}(t)$ it suffices to find one good $s$ isomorphic to $t$. Note that the Shannon entropy $H$ is a concave function and thus the weighted marginal entropy $\sum_{i=1}^{3}\theta_{i}H(P_{i})$ is a concave function of $P$. Therefore, the analysis of the maximization over $P$, being a convex program over an explicitly given domain, is usually straightforward.

It is not hard to see that the three flattening ranks $\operatorname{R}(t_{i})$ are among the support functionals, namely when we set $\theta$ to $(1,0,0)$, $(0,1,0)$ or $(0,0,1)$. Thus the support functionals can be thought of as interpolations between the three flattening ranks. Strassen proves in [Str91] the fundamental property that

[TABLE]

There are numerous examples where $\min_{\theta}\zeta^{\theta}(t)<\min_{i}\operatorname{R}(t_{i})$, of which we will see some later. It is not known whether the equality $\operatorname{\underaccent{\wtilde}{Q}}(t)=\min_{\theta}\zeta^{\theta}(t)$ holds in general. Strassen proved that equality holds for the family of tight tensors [Str91].333A tensor $t$ is called tight if for some choice of basis there are injective maps $\alpha_{1},\alpha_{2},\alpha_{3}$ such that for every $a\in\operatorname{supp}(t)$ it holds that $\alpha_{1}(a_{1})+\alpha_{2}(a_{2})+\alpha_{3}(a_{3})=0$. Tight tensors play an important role in the laser method for constructing matrix multiplication algorithms, as the laser method requires the outer structure of the intermediate tensor to be tight. We conclude that we have the following lower bound on the irreducibility in terms of the flattening ranks and the support functionals:

Proposition 24**.**

Let $t$ be a tensor. Then

[TABLE]

We will use the method of the support functionals to lower bound the irreversibility of some explicit tensors in Section 5.

Since we are primarily interested in using the support functionals to upper bound the asymptotic subrank, it is worth to observe that by the von Neumann minimax theorem we may for any fixed tensor $t$ express $\min_{\theta}\rho^{\theta}(t)$ in the concise form

[TABLE]

Indeed, the function $(\theta,P)\mapsto\sum_{i}\theta_{i}H(P_{i})$ is convex in $\theta$ and concave in $P$ so the minimax theorem allows us to swap the maximization over $P$ and the minimization over $\theta$. Moreover, $\min_{\theta}\sum_{i}\theta_{i}H(P_{i})$ is clearly attained when $\theta$ is one of the vertices $(1,0,0)$, $(0,1,0)$, $(0,0,1)$.

To further familiarize ourselves with the definition of $\zeta^{\theta}$, we discuss a simple example. For this example let $t$ be the so-called W-tensor $t=e_{1,2,2}+e_{2,1,2}+e_{2,2,1}$. We will upper bound $\zeta^{\theta}(t)$ for $\theta=(1/3,1/3,1/3)$. In the evaluation of $\rho^{\theta}(t)$ we take $s$ to be equal to $t$. (This turns out to be optimal in this case, since $t$ is tight [Str91].) Then the support of $s$ is the set

[TABLE]

Let $P\in\mathcal{P}(\operatorname{supp}(s))$ assign probability $a$ to $(1,2,2)$, probability $b$ to $(2,1,2)$ and probability $c$ to $(2,2,1)$. The function $P\mapsto\sum_{i}\tfrac{1}{3}H(P_{i})$ is concave and invariant under permuting the values of $a$, $b$ and $c$. Thus, the maximum $\max_{P\in\mathcal{P}(\operatorname{supp}(s))}\sum_{i}\tfrac{1}{3}H(P_{i})$ is attained by the symmetric probability distribution $P$ that assigns $a=b=c=1/3$ to each element in $\operatorname{supp}(s)$. Then each marginal distribution $P_{i}$ assigns probability $a$ to $1$ and probability $2a$ to $2$. Thus $\max_{P\in\mathcal{P}(\operatorname{supp}(s))}\sum_{i}\tfrac{1}{3}H(P_{i})$ equals the Shannon entropy of $P_{i}$, which is the binary entropy $h(1/3)=-\tfrac{1}{3}\log_{2}\tfrac{1}{3}-\tfrac{2}{3}\log_{2}\tfrac{2}{3}=0.918...$. We conclude that for the W-tensor $t$ it holds that the support functional $\zeta^{(1/3,1/3,1/3)}(t)$ is upper bounded by $\zeta^{(1/3,1/3,1/3)}(t)\leq 2^{h(1/3)}=1.88...$. Note that $1.88...$ is strictly smaller than the flattening rank $\operatorname{R}(t_{i})=2$. We remark that in fact in this case we know the asymptotic monomial subrank to be $\underaccent{\wtilde}{\mathrm{Q}}_{\mathrm{M}}(t)=\zeta^{(1/3,1/3,1/3)}(t)=2^{h(1/3)}=1.88...$ since this $t$ is tight. Since a discussion of tightness is not in the scope of this paper we refer to [Str91] for that.

5. Explicit barriers

We exhibit explicit barriers by computing lower bounds on the irreversibility of well-known intermediate tensors that play a crucial role in the best upper bounds on the matrix multiplication exponent $\omega$. These tensors are the small and big Coppersmith–Winograd tensors. Then we discuss the reduced polynomial multiplication tensors. Finally we discuss monomial irreversibility of structure tensors of finite group algebras and relations to the group-theoretic approach.

5.1. Irreversibility of Coppersmith–Winograd tensors

We now compute lower bounds for the irreversibility of the Coppersmith–Winograd tensors. As mentioned, we will use the support functionals of Strassen [Str91] in our computation to upper bound the asymptotic subrank.

(Upper bounds on the asymptotic subrank of complex tensors may be obtained, not only from the Strassen support functionals, but also from the quantum functionals. For the tensors in Theorem 25 and Theorem 28, however, it is known that the quantum functionals will give the same bound as the support functionals, since these tensors are free tensors [CVZ18, Section 4.3].)

Theorem 25** (Small Coppersmith–Winograd tensors [CW90, Section 6]).**

For any integer $q\geq 2$, the irreversibility of the small Coppersmith–Winograd tensor

[TABLE]

is lower bounded by

[TABLE]

Proof.

The rank of each flattening of $\mathrm{cw}_{q}$ equals $q+1$. Therefore, $\operatorname{\underaccent{\wtilde}{R}}(\mathrm{cw}_{q})\geq q+1$. To upper bound the asymptotic subrank $\operatorname{\underaccent{\wtilde}{Q}}(\mathrm{cw}_{q})$ one can upper bound the Strassen support functional with $\theta=(1/3,1/3,1/3)$ as in [CVZ18, Example 4.22] by

[TABLE]

We find that

[TABLE]

This proves the theorem. ∎

Remark 26**.**

If $q>2$, then the right-hand side of (51) is at least $2.02..$ See the table in Section 1 for more values. If $q=2$, however, then the right-hand side of (51) equals 2. Theorem 25 thus does not rule out using $\mathrm{cw}_{2}$ to prove that $\omega=2$. Indeed, as observed in [CW90, Section 11]), if $\omega(\langle 2\rangle,\mathrm{cw}_{2})=\log_{2}3$, then $\omega=2$.

Currently, the best upper bound we have on $\operatorname{\omega}(\langle 2\rangle,\mathrm{cw}_{q})$ is $\log_{2}(q+2)$. If $\omega(\langle 2\rangle,\mathrm{cw}_{q})=\log_{2}(q+2)$, then instead of (51) we get the better barrier

[TABLE]

The right-hand side of (54) has a minimum value of

[TABLE]

attained at $q=6$.

Remark 27**.**

The following computation serves as a sanity check for our barrier given by Theorem 18. Namely we see in an example how by putting some extra assumption the barrier given by Theorem 18 becomes tight. Coppersmith and Winograd in [CW90] used $\mathrm{cw}_{q}$ as an intermediate tensor in combination with the laser method and a certain outer structure. Here outer structure refers to the idea of viewing the intermediate tensor as a block tensor. The outer structure is the block-support of the intermediate tensor, see also [Blä13, Section 9]. For $\mathrm{cw}_{q}$ the outer structure is the W-tensor that we discussed in Section 4.2 which has asymptotic subrank $2^{h(1/3)}$. When we impose that we apply the laser method on $\mathrm{cw}_{q}$ with this outer structure to get an upper bound $\hat{\omega}\geq\omega$ we get the following better barrier via Theorem 18 or Corollary 19 and $\alpha=h(1/3)$ and $\beta=\tfrac{1}{3}\log_{2}(q)$:

[TABLE]

Some values of (56) are as follows, using (53) to get a bound on $\operatorname{\mathbf{i}}(\mathrm{cw}_{q})$:

[TABLE]

If in addition we assume that $\omega(\langle 2\rangle,\mathrm{cw}_{q})=\log_{2}(q+2)$, then we obtain the barrier

[TABLE]

Some values of (57) are as follows, using (54) to get a bound on $\operatorname{\mathbf{i}}(\mathrm{cw}_{q})$:

[TABLE]

with minimum value of 2.40… These barriers in fact match the upper bound

[TABLE]

that was obtained by Coppersmith and Winograd by applying the laser method in the way described above. Thus our sanity check succeeds. Other intermediate tensors with a given outer structure may be analyzed similarly.

Theorem 28** (Big Coppersmith–Winograd tensors [CW90, Section 7]).**

For any integer $q\geq 1$ the irreversibility of the big Coppersmith–Winograd tensor

[TABLE]

is lower bounded by

[TABLE]

where

[TABLE]

Proof.

First, we compute the asymptotic rank of $\mathrm{CW}_{q}$. One verifies directly that the matrix rank of each flattening of $\mathrm{CW}_{q}$ equals $q+2$, so $q+2\leq\operatorname{\underaccent{\wtilde}{R}}(\mathrm{CW}_{q})$. On the other hand, the well-known border rank upper bound $\operatorname{\underline{R}}(\mathrm{CW}_{q})\leq q+2$ implies that $\operatorname{\underaccent{\wtilde}{R}}(\mathrm{CW}_{q})\leq q+2$. We conclude that $\operatorname{\underaccent{\wtilde}{R}}(\mathrm{CW}_{q})=q+2$.

Second, we will upper bound the asymptotic subrank $\operatorname{\underaccent{\wtilde}{Q}}(\mathrm{CW}_{q})$ via the Strassen support functional $\zeta^{\theta}(\mathrm{CW}_{q})$ with $\theta=(1/3,1/3,1/3)$. Recall that $\zeta^{\theta}(\mathrm{CW}_{q})=2^{\rho^{\theta}(\mathrm{CW}_{q})}$ where

[TABLE]

We let $s=\mathrm{CW}_{q}$. Then the support of $\mathrm{CW}_{q}$ is given by

[TABLE]

We will now evaluate $\max_{P\in\mathcal{P}(\operatorname{supp}(\mathrm{CW}_{q}))}\sum_{i=1}^{3}\theta_{i}H(P_{i})$. We first observe that $\operatorname{supp}(\mathrm{CW}_{q})$ is symmetric under permutation of the three coordinates and under permutation of the elements of $[q]$. The function $P\mapsto\sum_{i=1}^{3}\theta_{i}H(P_{i})$ is concave. Thus by a simple convexity argument we may in our maximization, without loss of generality, consider only distributions $P$ that assign some probability $x$ to each of $(0,i,i)$, $(i,0,i)$ and $(0,i,i)$ for all $i\in[q]$, and probability $\tfrac{1}{3}-qx$ to each of $(0,0,q+1)$, $(0,q+1,0)$ and $(q+1,0,0)$. Under this simplification, we have that the average marginal entropy $\sum_{i=1}^{3}\theta_{i}H(P_{i})$ equals $f_{q}(x)$ as defined in the theorem statement. It remains to compute the maximum of $f_{q}(x)$ over all $0\leq x$ for which $3qx\leq 1$. From a calculus computation (easy to verify using computer algebra software) it follows that the maximum of $f_{q}(x)$ over all $0\leq x$ for which $3qx\leq 1$ is attained at

[TABLE]

Then $\operatorname{\underaccent{\wtilde}{Q}}(\mathrm{CW}_{q})\leq 2^{f_{q}(x_{0})}$. Thus we have

[TABLE]

which evaluates to the bound in the claim. ∎

Remark 29**.**

The lowest value of the right-hand side of (60) is $2.16..$ attained at $q=1$. See the table in Section 1 for more values and see Appendix A for code to compute any values.

5.2. Irreversibility of reduced polynomial multiplication tensors

The reduced polynomial multiplication tensors $t_{n}$ are an example of a natural family of tensors in which each tensor is irreversible, but where the irreversibility converges to 1 when $n$ goes to infinity. The irreversibility of the tensors $t_{n}$ can be computed directly from the computation of the asymptotic rank and asymptotic subrank of $t_{n}$ in [Str91, Theorem 6.7], which uses the support functionals. Namely, Strassen shows that

[TABLE]

and

[TABLE]

where

[TABLE]

and $g>1$ is the unique positive real solution to the equation

[TABLE]

Thus

[TABLE]

The limit $\lim_{n\to\infty}z(n)/n$ equals a constant, namely $0.84143...$ (see, e.g., [BCC*+*17a, Equation 4.11]). It follows that

[TABLE]

Thus $t_{n}$ is “reversible in the limit”. In Appendix A we provide code to compute the values of $\operatorname{\mathbf{i}}(t_{n})$ for any $n$.

5.3. Monomial irreversibility of structure tensors of finite group algebras

We now discuss irreversibility and monomial irreversibility in the context of the group-theoretic approach developed in [CU03]. This approach produces upper bounds on $\omega$ via intermediate tensors that are structure tensors of complex group algebras of finite groups. As defined earlier, let $\langle G\rangle_{\mathbb{F}}$ denote the structure tensor of the group algebra $\mathbb{F}[G]$ of the finite group $G$, in the standard basis, that is,

[TABLE]

The group-theoretic approach (in particular [CU03, Theorem 4.1]) produces an inequality of the form

[TABLE]

(the base field that is used in [CU03] is $\mathbb{C}$, in order to make the representation theory work) which ultimately (see [CU03, Eq. (1)]) leads to the bound

[TABLE]

where $\geq_{\mathrm{M}}$ and $\operatorname{\omega_{\mathrm{M}}}$ are the monomial restriction and monomial relative exponent defined in Section 2.4.

Now the monomial irreversibility barrier from Section 3.3 comes into play. Upper bounds on the monomial asymptotic subrank of $\langle G\rangle_{\mathbb{F}}$ have (using different terminology) been obtained in [BCC*+*17a, BCC*+*17b, Saw18]. Those upper bounds imply that $\langle G\rangle_{\mathbb{F}}$ is monomially irreversible for every nontrivial finite group $G$. Together with our results in Section 3.3 and the fact that the tensor $\langle G\rangle_{\mathbb{F}}$ is cyclically symmetric up to a permutation of the basis of one of the tensor legs, this directly leads to nontrivial barriers for the left-hand side of (66) for any fixed nontrivial group $G$, thus putting the work of [BCC*+*17a, BCC*+*17b, Saw18] in a broader context. We have not tried to numerically optimize the monomial irreversibility barriers for group algebras. We gave an example of a monomial barrier for $G=\mathbb{Z}/3\mathbb{Z}$ in Example 11.

It should be stressed again that although these results rule out using monomial restrictions from powers of any one single group, part of the hope of the group-theoretic approach is to use a family of groups (such as the symmetric group $S_{n}$ for increasing $n$) which is not just powers of a single starting group. For families of abelian groups of bounded exponent (even if they are not powers of a single group), [BCC*+*17a] rules out STPP constructions reaching $\omega=2$, and for certain families of nilpotent groups (again, even if they are not just powers of a single group) [BCC*+*17b] does similarly.

The results of [BCC*+*17a], [BCC*+*17b] and [Saw18], in fact, show slightly more than the aforementioned barriers for monomial restriction. While, over arbitrary fields, they only rule out monomial restrictions, over fields of bad characteristic (e.g., characteristic $p$ when $G$ is a $p$-group satisfying the relevant conditions of their theorems) they also rule out arbitrary degenerations. This is because they show slice rank upper bounds, the slice rank of a diagonal tensor equals its rank, and having slice rank at most $r$ is a Zariski-closed condition (proved in [TS16]).

Finally, we mention that the irreversibility barrier (rather than the monomial irreversibility barrier) does not rule out obtaining $\omega=2$ via $\langle G\rangle_{\mathbb{C}}$. Namely, $\langle G\rangle_{\mathbb{C}}$ is isomorphic to a direct sum of matrix multiplication tensors, $\langle G\rangle_{\mathbb{C}}\cong\bigoplus_{i}\langle d_{i},d_{i},d_{i}\rangle$ and, therefore, the irreversibility satisfies $\operatorname{\mathbf{i}}(\langle G\rangle_{\mathbb{C}})=(\log_{2}\sum_{i}d_{i}^{\omega})/(\log_{2}\sum_{i}d_{i}^{2})$. Thus, if $\omega=2$, then $\langle G\rangle_{\mathbb{C}}$ is reversible.

6. Outlook for further barriers

In Section 5 we used the support functionals of Section 4 to upper bound the asymptotic subrank and thus lower bound the irreversibility of intermediate tensors. There are two other approaches to do this that we will discuss now. These approaches are at least as powerful as the support functionals for upper bounding asymptotic subrank, and we expect that there are examples where they perform better.

6.1. The quantum functionals

For tensors over the complex numbers (i.e., when $\mathbb{F}=\mathbb{C}$) we have a much deeper understanding of the theory of upper bounds on the asymptotic subrank. For any probability vector $\theta\in\mathbb{R}^{3}$ the quantum functional $F^{\theta}$ introduced in [CVZ18], or rather its logarithm, is defined as

[TABLE]

where $\Pi(t)$ is the moment polytope of $t$. It is outside the scope of this paper to go into the definition of the moment polytope. For that and a thorough discussion of all the connections to representation theory, invariant theory and quantum information theory we refer to [CVZ18]. Here we will discuss the properties of the quantum functionals that are important for upper bounding the asymptotic subrank. The first crucial fact proven in [CVZ18] is that the quantum functionals are in the asymptotic spectrum of all complex tensors \Delta(\{\textnormal{tensors over \mathbb{C}}\}), which we discussed in Section 4.1. We conclude that we can lower bound irreversibility in terms of the flattening ranks and the quantum functionals as follows:

Proposition 30**.**

Let $t$ be a tensor over the complex numbers. Then

[TABLE]

Comparing to Section 4.2, it is proved in [CVZ18] that the quantum functionals are at least as powerful as the support functionals when it comes to upper bounding the asymptotic subrank. Namely, for any tensor $t$ over the complex numbers holds that $F^{\theta}(t)\leq\zeta^{\theta}(t)$. It is an open problem whether this inequality can be strict. We note that, as opposed to the support functionals, the quantum functionals are defined as convex programs. It is an open problem whether the quantum functionals are efficiently computable.

6.2. Asymptotic slice rank

We finish Section 5 by discussing the asymptotic slice rank as a method to upper bound asymptotic subrank, and the relations to the support functionals and the quantum functionals. The slice rank (introduced by Tao [Tao16] in the context of the cap set problem) of a tensor $t$ is the smallest number $r$ such that $t$ can be written as a sum of $r$ slice rank one tensors. A slice rank one tensor is a tensor for which there is an $i\in[3]$ such that the flattening $t_{i}$ has matrix rank one. The asymptotic subrank, the slice rank and the support functionals are related in the following way:

[TABLE]

(See [CVZ18].) Thus, in an asymptotic fashion, the slice rank upper bounds the asymptotic subrank and hence lower bounds irreducibility. Any analysis of $\limsup_{n}\operatorname{slicerank}(t^{\otimes n})^{1/n}$ that we are aware of in the literature boils down to evaluating $\min_{\theta}\zeta^{\theta}(t)$. In particular, we are not aware of any example for which the right-most inequality in (68) is strict.

In fact, for oblique444A tensor $t\in\mathbb{F}^{n_{1}}\otimes\mathbb{F}^{n_{2}}\otimes\mathbb{F}^{n_{3}}$ is called oblique if the support $\operatorname{supp}(t)\in[n_{1}]\times[n_{2}]\times[n_{3}]$ in some basis is an antichain in the product of the natural orders on the $[n_{i}]$. The matrix multiplication tensors $\langle a,b,c\rangle$ are examples of oblique tensors. tensors the right-most inequality is an equality [TS16] (see also [Zui18, Section 4.6]) and, as mentioned in Section 4.2, for tight tensors both inequalities are equalities [Str91]. Over the complex numbers, the quantum functionals play a special role in this comparison. As we mentioned in Section 6.1, the quantum functionals satisfy $F^{\theta}(t)\leq\zeta^{\theta}(t)$. It was shown in [CVZ18] that the minimum over the parameter $\theta$ equals the asymptotic slice rank. We thus have

[TABLE]

For free555A tensor $t$ is called free if in some basis any two different $a,b\in\operatorname{supp}(t)$ differ in at least two entries. Every tight tensor is oblique and every oblique tensor is free. tensors the right-most inequality in (69) is an equality [CVZ18].

Appendix A Code to verify the numerical examples

The following Mathematica code generates the barrier values in the tables in Section 1 to arbitrary precision and for arbitrary parameters $q$ and $n$.

A.1. Small Coppersmith–Winograd tensor $\mathrm{cw}_{q}$

In[1]:= Table[ 2 Log2[q + 1]/(Log2[3] - 2/3 + (2/3) Log2[q]), {q, 2, 7}] // N

Out[1]= {2., 2.02538, 2.06244, 2.09627, 2.12549, 2.15064}

A.2. Big Coppersmith–Winograd tensor $\mathrm{CW}_{q}$

In[1]:= g[b_, q_] := -((2 b q Log[2 b])/ Log[2]) - ((1 - 3 b q) Log[1/3 (1 - 3 b q)])/( 3 Log[2]) + ((-b q - 2/3 (1 - 3 b q)) Log[b q + 2/3 (1 - 3 b q)])/ Log[2]

In[2]:= f[q_] := Piecewise[{{g[ q/(2 (-4 + q^2)) + 1/6 Sqrt[(32 + q^2)/(-4 + q^2)^2] /. {q -> 1}, 1], q == 1}, {g[1/9, q], q == 2}, {g[-((-3 q + Sqrt[32 + q^2])/(6 (-4 + q^2))), q], q >= 3}}]

In[3]:= Table[2 Log2[q + 2]/f[q], {q, 1, 6}] // N

Out[3]= {2.16805, 2.17795, 2.19146, 2.20551, 2.21913, 2.23201}

A.3. Reduced polynomial multiplication $t_{n}$

In[1]:= g[n_] := g[n] = g /. FindRoot[1/(g-1)-n/(g^n-1)==(n-1)/3,{g,1+2/n}]

In[2]:= z[n_] := (g[n]^n - 1)/(g[n] - 1) g[n]^(-2 (n - 1)/3)

In[3]:= Table[2 Log2[n]/Log2[z[n]], {n, 2, 7}]

Out[3]= {2.17795, 2.16805, 2.15949, 2.15237, 2.14641, 2.14135}

Acknowledgements

MC acknowledges financial support from the European Research Council (ERC Grant Agreement No. 337603 and 81876) and VILLUM FONDEN via the QMATH Centre of Excellence (Grant No. 10059). This research was supported by the National Research, Development and Innovation Fund of Hungary within the Quantum Technology National Excellence Program (Project Nr. 2017-1.2.1-NKP-2017-00001) and via the research grants K124152, KH129601 (PV). This material is based upon work directly supported by the National Science Foundation Grant No. DMS-1638352 and indirectly supported by the National Science Foundation Grant No. CCF-1900460. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation (JZ).