Sum-of-Squares Optimization and the Sparsity Structure of Equiangular Tight Frames

TL;DR

This paper explores the connection between equiangular tight frames and sum-of-squares optimization, deriving new bounds on ETF sparsity and highlighting optimal sparsity in Steiner ETFs related to finite projective planes.

Contribution

The authors generalize previous calculations to establish new bounds on the sparsity of real and complex ETFs, and identify Steiner ETFs as optimally sparse in certain matrix inequalities.

Findings

Steiner ETFs achieve tight bounds on sparsity.

New bounds on ETF sparsity are derived.

Open problems for further generalizations are proposed.

Abstract

Equiangular tight frames (ETFs) may be used to construct examples of feasible points for semidefinite programs arising in sum-of-squares (SOS) optimization. We show how generalizing the calculations in a recent work of the authors' that explored this connection also yields new bounds on the sparsity of (both real and complex) ETFs. One corollary shows that Steiner ETFs corresponding to finite projective planes are optimally sparse in the sense of achieving tightness in a matrix inequality controlling overlaps between sparsity patterns of distinct rows of the synthesis matrix. We also formulate several natural open problems concerning further generalizations of our technique.

Tables

Table 1. Table 1: Spark Lower Bound Comparison. We tabulate three lower bounds on the spark of ETFs of N 𝑁 N vectors in ℝ r superscript ℝ 𝑟 \mathbb{R}^{r} formed as the Naimark complements of ETFs belonging to infinite families with N ∼ r 3 / 2 similar-to 𝑁 superscript 𝑟 3 2 N\sim r^{3/2} . We express the parameters of these Naimark complements in terms of an integer parameter q 𝑞 q for each family. (There is a fifth construction based on difference sets in finite abelian groups having this scaling [ 8 ] , but its parameters are the same as those of Steiner ETFs built from finite affine planes, so we omit it from the table.) The first two bounds are from prior work described in Propositions  1.15 and 1.16 , and the last is from our Corollary  3.3 . Since the spark is always an integer, the ceiling function may be applied to any valid lower bound on it to obtain a stronger lower bound. Thus, when the bound expressions do not simplify to integer values in terms of q 𝑞 q , we take a partial fraction decomposition and give a bound holding for sufficiently large q 𝑞 q .

Construction	$𝑵$	$𝒓$	Gershgorin	NERF	Our Bound
Steiner: Affine [17]	$q^3+2q^2$	$q^2+q$	$q^2+q$	$q^2+q-1$	$q^2+q$
Steiner: Projective [17]	$q^3+3q^2+3q+2$	$q^2+q+1$	$q^2+2q+2$	$q^2+3q+1$	$q^2+3q+2$
Polyphase BIBD [13]	$q^3+1$	$q^2-q+1$	$q^2+1$	$q^2+q-1$	$q^2+q$
Hyperovals [16]	$q^3+q^2-q$	$q^2+q-1$	$q^2$	$q^2-q+3$	$q^2-q+4$

Equations

$${\mathcal{C}}^N:=\mathsf{conv}\left(\left\{\mathbf{x}{\mathbf{x}}^{\top }:\mathbf{x}\in \{\pm 1{\}}^N\right\}\right)\subset {\mathbb{R}}_{\mathsf{sym}}^{N\times N}.$$

$${\mathcal{E}}^N:=\left\{\mathbf{X}\in {\mathbb{R}}_{\mathsf{sym}}^{N\times N}:\mathbf{X}⪰0,\mathsf{diag}(\mathbf{X})=1\right\}\supseteq {\mathcal{C}}^N.$$

$${\mathcal{E}}^N={\mathcal{E}}_2^N⊋{\mathcal{E}}_4^N⊋\cdots ⊋{\mathcal{E}}_{N+\text{mathbbold}1\{N\text{ odd}\}}^N={\mathcal{C}}^N.$$

$${\mathsf{pert}}_K(\mathbf{X}):=\left\{\mathbf{A}\in {\mathbb{R}}^d:\exists t>0\text{ with }\mathbf{X}\pm t\mathbf{A}\in K\right\}.$$

$$\mathbf{Y}:=\mathsf{vec}(\mathbf{X})\mathsf{vec}(\mathbf{X}{)}^{\top }+\frac{N^2(1-\frac{1}{r})}{\frac{r(r+1)}{2}-N}\mathbf{P}$$

$${\tilde{\mathcal{E}}}^N:=\left\{\mathbf{X}\in {\mathbb{C}}_{\mathsf{herm}}^{N\times N}:\mathbf{X}⪰0,\mathsf{diag}(\mathbf{X})=1\right\}.$$

$$⟨\mathbf{X},\mathbf{Y}{⟩}_F$$

$$\Vert \mathbf{X}{\Vert }_F$$

$$\underset{\genfrac{}{}{0ex}{}{1\le i,j\le N}{i\ne j}}{\max }|⟨{\mathbf{v}}_i,{\mathbf{v}}_j⟩|\ge \sqrt{\frac{N-r}{r(N-1)}},$$

$$N\leq\left\{\begin{array}[]{ccl}r(r+1)/2&:&\mathbb{F}=\mathbb{R}\\ r^{2}&:&\mathbb{F}=\mathbb{C}\end{array}\right\}.$$

$$|\mathbf{X}{|}^{\odot 2}=(1-{\alpha }^2){\mathbf{I}}_N+{\alpha }^{2}11^{\top },$$

$$\mathsf{spark}(\mathbf{V}):=\underset{\begin{array}{c}\mathbf{x}\in {\mathbb{R}}^N\setminus \{0\}\\ \mathbf{V}\mathbf{x}=0\end{array}}{\min }\Vert \mathbf{x}{\Vert }_0=\underset{\mathbf{x}\in \mathsf{row}(\mathbf{V}{)}^{\perp }\setminus \{0\}}{\min }\Vert \mathbf{x}{\Vert }_0.$$

$$\mathsf{sparsity}(\mathbf{V}):=\underset{\mathbf{x}\in \mathsf{row}(\mathbf{V})\setminus \{0\}}{\min }\Vert \mathbf{x}{\Vert }_0.$$

$$\mathsf{sparsity}(\mathbf{V})=\mathsf{spark}({\mathbf{V}}^{\prime }).$$

$$\mathsf{Spark}(\mathbf{V})=1+\max \{k:\text{every }k\times k\text{ principal minor of }{\mathbf{V}}^{\top }\mathbf{V}\text{ is non-singular}\}.$$

$$\mathsf{spark}(\mathbf{V})\ge N{\left(1+\frac{(N-r)(N-r-1)}{N-1}\right)}^{-1}.$$

$$\mathcal{P}\mathbf{A}:=\underset{\mathbf{B}\in {\mathsf{pert}}_{{\tilde{\mathcal{E}}}^N}(\mathbf{X})}{\mathrm{argmin}}\frac{1}{2}\Vert \mathbf{A}-\mathbf{B}{\Vert }_F^2.$$

$${\mathsf{pert}}_{{\tilde{\mathcal{E}}}^N}(\mathbf{X})=\left\{{\mathbf{V}}^{\ast }\mathbf{H}\mathbf{V}:\mathbf{H}\in {\mathbb{C}}_{\mathsf{herm}}^{r\times r},{\mathbf{v}}_i^{\ast }\mathbf{H}{\mathbf{v}}_i=0\text{ for all }i\in [N]\right\}.$$

$$\mathcal{P}\bm{A}=\frac{r^{2}}{N^{2}}\Bigg{(}\bm{X}\bm{A}\bm{X}-\sum_{i,j=1}^{N}(|\bm{X}|^{\odot 2})^{-1}_{ij}(\bm{x}_{i}^{*}\bm{A}\bm{x}_{i})\bm{x}_{j}\bm{x}_{j}^{*}\Bigg{)}.$$

$$\mathcal{P}\mathbf{A}$$

$${\mathbf{H}}^{\star }(\mathbf{A})$$

$$\mathsf{obj}(\mathbf{H};\mathbf{A})$$

$$=\frac{1}{2}\Vert \mathbf{A}{\Vert }_F^2+\frac{N^2}{2r^2}\Vert \mathbf{H}{\Vert }_F^2-⟨\mathbf{V}\mathbf{A}{\mathbf{V}}^{\ast },\mathbf{H}⟩.$$

$$L(\mathbf{H},\mathbf{\gamma };\mathbf{A})$$

$$=\frac{1}{2}\Vert \mathbf{A}{\Vert }_F^2+\frac{N^2}{2r^2}\Vert \mathbf{H}{\Vert }_F^2-⟨\mathbf{V}\mathbf{A}{\mathbf{V}}^{\ast }+\sum _{i=1}^N{\gamma }_i{\mathbf{v}}_i{\mathbf{v}}_i^{\ast },\mathbf{H}⟩.$$

$${\mathbf{H}}^{\star }(\mathbf{A})=\mathbf{V}\mathbf{A}{\mathbf{V}}^{\ast }+\sum _{i=1}^N{\gamma }_i(\mathbf{A}){\mathbf{v}}_i{\mathbf{v}}_i^{\ast }$$

$$\sum _{j=1}^N|⟨{\mathbf{v}}_i,{\mathbf{v}}_j⟩{|}^2{\gamma }_j(\mathbf{A})=-{\mathbf{v}}_i^{\ast }\mathbf{V}\mathbf{A}{\mathbf{V}}^{\ast }{\mathbf{v}}_i\text{ for }i\in [N],$$

$${\gamma }_i(\mathbf{A})=-\sum _{j=1}^N(|\mathbf{X}{|}^{\odot 2}{)}_{ij}^{-1}{\mathbf{v}}_j^{\ast }\mathbf{V}\mathbf{A}{\mathbf{V}}^{\ast }{\mathbf{v}}_j.$$

$$(|\mathbf{V}{|}^{\odot 2})(|\mathbf{X}{|}^{\odot 2}{)}^{-1}(|\mathbf{V}{|}^{\odot 2}{)}^{\top }⪯{\mathbf{I}}_r,$$

$$(|\mathbf{V}{|}^{\odot 2}{)}^{\top }(|\mathbf{V}{|}^{\odot 2})⪯|\mathbf{X}{|}^{\odot 2}.$$

$$\mathcal{P}\mathbf{A}$$

$${\mathcal{P}}_1\mathbf{A}$$

$${\mathcal{P}}_2\mathbf{A}$$

$$⟨\mathbf{A},{\mathcal{P}}_1\mathbf{A}⟩$$

Full text

1 Introduction

One of the most important objects in combinatorial optimization is the cut polytope, the convex set of matrices

[TABLE]

The cut polytope has a rich discrete geometry [7] describing the solution space of the problem of finding the largest cut in a graph, which amounts to maximizing a linear function over $\mathscr{C}^{N}$. By the classical result of Karp [21], this problem cannot be solved in polynomial time unless $\mathsf{P}=\mathsf{NP}$. Therefore, relaxations of $\mathscr{C}^{N}$ to larger but more algorithmically tractable convex sets have been proposed, perhaps the best-known of which is the relaxation to the (real) elliptope

[TABLE]

There is extensive literature both on the geometry of $\mathscr{E}^{N}$ (thoroughly described in [7]), and on approximating optimization over $\mathscr{C}^{N}$ by optimization over $\mathscr{E}^{N}$ (e.g. [19, 26, 11, 22]).

Every $\bm{X}\in\mathscr{E}^{N}$ is the Gram matrix of unit vectors, $X_{ij}=\langle\bm{v}_{i},\bm{v}_{j}\rangle$ for some $\bm{v}_{1},\dots,\bm{v}_{N}\in\mathbb{R}^{r}$ with $r\colonequals\mathsf{rank}(\bm{X})\leq N$. The boundary of $\mathscr{E}^{N}$ consists of $\bm{X}$ having $r<N$. In the language of frame theory, these boundary points are the Gram matrices of overcomplete unit norm frames. We will explore applications of geometric results about $\mathscr{E}^{N}$ to the following types of structured frames.

Definition 1.1**.**

Unit vectors $\bm{v}_{1},\dots,\bm{v}_{N}\in\mathbb{C}^{r}$ with $\bm{X}=(\langle\bm{v}_{i},\bm{v}_{j}\rangle)_{i,j=1}^{N}$ form a unit norm tight frame (UNTF) if any of the following equivalent conditions hold.

$\sum_{i=1}^{N}\bm{v}_{i}\bm{v}_{i}^{*}=\frac{N}{r}\bm{I}_{N}$. 2. 2.

The nonzero eigenvalues of $\bm{X}$ all equal $\frac{N}{r}$. 3. 3.

$\|\bm{X}\|_{F}^{2}=\sum_{i=1}^{N}\sum_{j=1}^{N}|X_{ij}|^{2}=\frac{N^{2}}{r}$.

*The $\bm{v}_{i}$ form an equiangular tight frame (ETF) if moreover there exists $\alpha\in[0,1]$ such that $|X_{ij}|=|\langle\bm{v}_{i},\bm{v}_{j}\rangle|=\alpha$ whenever $i\neq j$. *

When moreover $\bm{v}_{i}\in\mathbb{R}^{r}$, we have $\bm{X}\in\mathscr{E}^{N}$, and such points form an interesting subset of the elliptope’s boundary. In both the real and complex cases, UNTFs and ETFs have been studied in great detail previously (e.g. [3, 4, 27, 5, 15]).

The set $\mathscr{E}^{N}$, however, is only the first of a sequence of tightening relaxations of $\mathscr{C}^{N}$, described by the sum-of-squares (SOS) hierarchy. This gives sets we call degree $d$ generalized elliptopes $\mathscr{E}_{d}^{N}$ for positive even integers $d$.

Definition 1.2**.**

$\mathscr{E}_{d}^{N}$* is the set of $\bm{X}\in\mathbb{R}^{N\times N}_{\mathsf{sym}}$ for which there exists $\bm{Y}\in\mathbb{R}^{N^{d/2}\times N^{d/2}}_{\mathsf{sym}}$ where, identifying indices of $\bm{Y}$ with elements of $[N]^{d/2}$, the following conditions hold.*

$Y_{\bm{i}\bm{j}}=1$* whenever all indices occur an even number of times across $\bm{i}$ and $\bm{j}$.* 2. 2.

$Y_{\bm{i}\bm{j}}$* depends only on the set of indices occuring an odd number of times across $\bm{i}$ and $\bm{j}$.* 3. 3.

$Y_{(1,\dots,1,i)(1,\dots,1,j)}=X_{ij}$* for all $i,j\in[N]$.* 4. 4.

$\bm{Y}\succeq\bm{0}$.

We have $\mathscr{C}^{N}\subseteq\mathscr{E}_{d}^{N}$ since both sets are convex and $\bm{x}\bm{x}^{\top}\in\mathscr{E}_{d}^{N}$ for $\bm{x}\in\{\pm 1\}^{N}$ by taking $\bm{Y}=(\bm{x}^{\otimes d/2})(\bm{x}^{\otimes d/2})^{\top}$. The following inclusions also hold (these non-trivial facts are the combined results of [10] and [23]):

[TABLE]

In the recent work [2], the authors showed that the Gram matrix of a real ETF belongs to $\mathscr{E}_{4}^{N}$ if and only if $N<\frac{r(r+1)}{2}$. The construction involves the following general concept from convex geometry.

Definition 1.3**.**

For a closed convex set $K\subseteq\mathbb{R}^{d}$ and $\bm{X}\in K$, the perturbation of $\bm{X}$ in $K$ is the subspace

[TABLE]

We then showed in [2] that one may take

[TABLE]

for $\bm{P}$ the projection matrix to $\mathsf{vec}(\mathsf{pert}_{\mathscr{E}_{2}^{N}}(\bm{X}))$. To verify the conditions of Definition 1.2, we computed $\bm{P}$ for all real ETFs.

In this paper, we give a self-contained presentation of this calculation for perturbation subspaces in the complex generalization of the elliptope,

[TABLE]

Using that this projection operator is positive semidefinite (psd), we derive inequalities in degree 4 polynomials of the entries of (real or complex) ETF vectors, which translate to new inequalities controlling the sparsity and spark of ETF vectors.

1.1 Background

1.1.1 Linear Algebra

We first introduce some notations for standard linear-algebraic tools we will use. Denote by $\mathbb{C}^{N\times N}_{\mathsf{herm}}$ the set of $N\times H$ Hermitian matrices. We recall the usual Hilbert space structure on $\mathbb{C}^{N\times N}_{\mathsf{herm}}$.

Definition 1.4**.**

For $\bm{X},\bm{Y}\in\mathbb{C}^{N\times N}_{\mathsf{herm}}$, define the Frobenius inner product and associated Frobenius norm as

[TABLE]

We will also use the following entrywise transformations of matrices.

Definition 1.5**.**

*For $\bm{A}\in\mathbb{C}^{a\times b}$, let $|\bm{A}|$ denote the entrywise absolute value of $\bm{A}$. *

Definition 1.6**.**

*For $\bm{A}\in\mathbb{C}^{a\times b}$, let $\bm{A}^{\odot k}$ denote the entrywise $k$th power of $\bm{A}$ for $k\in\mathbb{Z}$. *

1.1.2 Combinatorics

We present some definitions of combinatorial objects involved in the construction of Steiner ETFs that we will study in greater detail in the sequel (the construction itself is discussed in Section 1.1.5). A full motivation of these definitions is beyond the scope of the present paper; we refer the reader to [6] for further information and references.

The following highly structured graphs are intimately related to general real ETFs, and will also play a role in analyzing our results for the special case of Steiner ETFs.

Definition 1.7**.**

*A strongly regular graph with parameters $(v,k,\lambda,\mu)$, abbreviated $\mathsf{srg}(v,k,\lambda,\mu)$, is a graph $G=(V,E)$ for which $|V|=v$, $G$ is $k$-regular, and every pair of distinct vertices in $G$ have $\lambda$ common neighbors if they are adjacent and $\mu$ common neighbors if they are not adjacent. *

The first type of combinatorial design we will be interested in for ETF constructions is the following generalization of the incidence structure of finite geometries.

Definition 1.8**.**

*A $(t,k,v)$-Steiner system is a pair $(S,\mathcal{B})$ where $|S|=v$ and $\mathcal{B}\subseteq 2^{S}$, such that for all $B\in\mathcal{B}$ we have $|B|=k$, and for any $C\subseteq S$ with $|C|=t$, there exists a unique $B\in\mathcal{B}$ such that $C\subseteq B$. The elements of $S$ are called points, and the elements of $\mathcal{B}$ are called blocks. A Steiner system is completely specified by its incidence matrix $\bm{N}\in\{0,1\}^{v\times|\mathcal{B}|}$, with entries equal to 1 when the block index contains the point index and equal to 0 otherwise. Also associated to a Steiner system is its block intersection graph, a graph on vertex set $\mathcal{B}$ where $B,B^{\prime}\in\mathcal{B}$ are adjacent if and only if $B\cap B^{\prime}\neq\emptyset$. *

The following are basic combinatorial results on Steiner systems with $t=2$.

Proposition 1.9**.**

*Let $(S,B)$ be a $(2,k,v)$-Steiner system. Then, $b\colonequals|\mathcal{B}|=\frac{v(v-1)}{k(k-1)}$, and each point is contained in $\rho\colonequals\frac{v-1}{k-1}$ blocks (typically this is denoted $r$ in the combinatorics literature, which we adjust to avoid conflict with our notation for frame dimensions). *

Important examples of Steiner systems with $t=2$ are given by finite projective and affine planes.

Definition 1.10**.**

A $(2,k,v)$-Steiner system $(S,\mathcal{B})$ is called a finite projective plane if for any distinct $B,B^{\prime}\in\mathcal{B}$ we have $|B\cap B^{\prime}|=1$, and there exist $w,x,y,z\in S$ such that no $B\in\mathcal{B}$ contains more than two of these points. The number $k-1$ is called the order of a finite projective plane.

*A $(2,k,v)$-Steiner system $(S,\mathcal{B})$ is called an finite affine plane if $|B|\geq 2$ for all $B\in\mathcal{B}$, for any $B\in\mathcal{B}$ and $s\in S\setminus B$ there exists a unique $B^{\prime}\in\mathcal{B}$ with $s\in B^{\prime}$ and $B\cap B^{\prime}=\emptyset$, and there exist $x,y,z\in S$ such that no $B\in\mathcal{B}$ contains all three of these points. The number $k$ is called the order of a finite affine plane. *

Proposition 1.11**.**

*A finite projective plane of order $q$ is a $(2,q+1,q^{2}+q+1)$-Steiner system. A finite affine plane of order $q$ is a $(2,q,q^{2})$-Steiner system. *

The most common specific constructions of finite affine planes of order $q$ have as points the elements of $\mathbb{F}^{2}$ for a finite field $\mathbb{F}$ with $|\mathbb{F}|=q$, and as lines the one-dimensional affine subspaces in this vector space over $\mathbb{F}$. Analogously, the most common specific constructions of finite projective planes of order $q$ have as points the one-dimensional linear subspaces of $\mathbb{F}^{3}$, and as blocks the two-dimensional linear subspaces of $\mathbb{F}^{3}$. These give finite projective and affine planes of any order $q$ equal to a prime power; the question of whether there exist finite projective or affine planes of other orders is a longstanding open problem in combinatorics.

The other ingredient in the Steiner ETF construction is the following class of matrices, which describe certain structured orthonormal bases of $\mathbb{C}^{N}$.

Definition 1.12**.**

*A (complex) Hadamard matrix is a matrix $\bm{H}\in\mathbb{C}^{N\times N}$ such that $|H_{ij}|=1$ for all $i,j\in[N]$, and $\bm{H}\bm{H}^{*}=N\bm{I}_{N}$. *

1.1.3 Equiangular Tight Frames: General Theory

We next review some background on the theory of ETFs. ETFs do not exist in all pairs of dimension $(N,r)$, and the known constructions are mostly based on exceptional combinatorial structures. Determining for which dimensions ETFs do exist is thus an important open problem with implications ranging many topics in combinatorics. Real-valued ETFs are better understood than the more general complex-valued ETFs, and the techniques for treating these two variants are often somewhat different. In particular, there are several correspondences between real ETFs and strongly regular graphs [18, 12] which allow combinatorial constructions to immediately yield real ETFs, while no such general connections are known to hold for complex ETFs. More comprehensive references on these aspects of the theory of ETFs include [27, 5, 15].

The following important result shows that ETFs are extremal among UNTFs in the sense of worst-case coherence. Moreover, when an ETF exists, $\alpha$ is determined by $N$ and $r$.

Proposition 1.13** (Welch Bound [29]).**

If $\bm{v}_{1},\dots,\bm{v}_{N}\in\mathbb{C}^{r}$ with $\|\bm{v}_{i}\|_{2}=1$, then

[TABLE]

*with equality if and only if $\bm{v}_{1},\dots,\bm{v}_{N}$ form an ETF. *

The other important limitation on ETFs that we will be concerned with is on the maximum number of vectors in $\mathbb{R}^{r}$ or $\mathbb{C}^{r}$ that can form an ETF. We include the proof of this result, since it is short and involves a matrix associated to an ETF that will play an important role in our later calculations.

Proposition 1.14** (Gerzon Bound [24]).**

Let $\mathbb{F}\in\{\mathbb{R},\mathbb{C}\}$, and let $\bm{v}_{1},\dots,\bm{v}_{N}\in\mathbb{F}^{r}$ form an ETF with coherence $\alpha<1$. Then,

[TABLE]

Proof.

Let $\mathbb{M}=\mathbb{R}^{r\times r}_{\mathsf{sym}}$ if $\mathbb{F}=\mathbb{R}$ and $\mathbb{M}=\mathbb{C}^{r\times r}_{\mathsf{herm}}$ if $\mathbb{F}=\mathbb{C}$. For all $i\neq j$, $\langle\bm{v}_{i}\bm{v}_{i}^{*},\bm{v}_{j}\bm{v}_{j}^{*}\rangle=\alpha^{2}$. Thus, the Gram matrix of the $\bm{v}_{i}\bm{v}_{i}^{*}\in\mathbb{M}$ is

[TABLE]

which is non-singular since $\alpha<1$. The $\bm{v}_{i}\bm{v}_{i}^{*}$ are then linearly independent, so $N\leq\mathsf{dim}(\mathbb{M})$, and the result follows.

Lastly, an involution of ETFs called the Naimark complement will play an important role in some of our reasoning. In the sequel, we will sometimes identify an ETF $\bm{v}_{1},\dots,\bm{v}_{N}\in\mathbb{C}^{r}$ with its so-called synthesis matrix $\bm{V}\in\mathbb{C}^{r\times N}$, the matrix whose columns are the $\bm{v}_{i}$. Since the rows of $\bm{V}$ are orthogonal and have equal norm, $\bm{V}$ may be completed by further rows to form a scaled orthogonal matrix. The added rows form a matrix, which we denote $\bm{V}^{\prime}\in\mathbb{C}^{(N-r)\times N}$, which also has orthogonal rows of equal norm and columns of equal norm, and hence, suitably scaled, is the synthesis matrix of a new UNTF consisting of $N$ vectors in $\mathbb{C}^{N-r}$. Moreover, one may verify that this UNTF is in fact another ETF (with different coherence). Choosing among the possible completions to an orthogonal matrix appropriately, we may guarantee that the Naimark complement is an involution of ETFs: $\bm{V}^{\prime\prime}=\bm{V}$ for every ETF $\bm{V}$.

1.1.4 Equiangular Tight Frames: Sparsity and Spark

Two quantities of interest for ETFs are the sparsity, the number of non-zero entries of the ETF vectors or the synthesis matrix, and the spark, the smallest number of ETF vectors involved in a non-trivial linear dependency. The conventional definition of the latter is

[TABLE]

The natural dual measure of sparsity, sometimes called cospark, is

[TABLE]

This is a stronger notion that merely the sparsity of the synthesis matrix $\bm{V}$ itself, but our results are most naturally stated in these terms. One convenience of working with this notion of sparsity is that, since the Naimark complement exchanges the row space and kernel of an ETF synthesis matrix, our sparsity is in fact merely the spark of the Naimark complement:

[TABLE]

We will later be concerned with the quality of certain lower bounds on the spark (and thus, via the above remark, the sparsity) of ETFs, so we briefly review the existing general lower bounds we are aware of. The simplest spark lower bound is based on an argument via the Gershgorin circle theorem [9], which we present below.

Proposition 1.15**.**

*If $\bm{V}$ is an ETF with coherence $\alpha$, then $\mathsf{spark}(\bm{V})\geq 1+\alpha^{-1}$. *

Proof.

One may easily check the following alternative formulation of the spark:

[TABLE]

It then suffices to show that every $\alpha^{-1}\times\alpha^{-1}$ principal minor is non-singular. Such a minor is a $\alpha^{-1}\times\alpha^{-1}$ matrix whose diagonal entries are 1 and whose off-diagonal entries are $\alpha$ in magnitude. By the Gershgorin circle theorem, the eigenvalues of such a matrix are all at least $1-\frac{\alpha^{-1}-1}{\alpha^{-1}}>0$, hence it is non-singular.

Another lower bound on the spark follows from considerations of so-called numerically erasure-robust frames (NERFs) in [14].333We thank Dustin Mixon for bringing this connection to our attention.

Proposition 1.16**.**

If $\bm{V}$ is an ETF of $N$ vectors in $\mathbb{R}^{r}$, then

[TABLE]

Proof.

This follows directly from Theorem 5 of [14].

Our result on spark will sharpen Proposition 1.16, so we will be interested in cases where this result is superior to the Gershgorin circle argument.

To this end, consider a scaling regime where $N-r\sim r^{\beta}$. Then, the NERF spark lower bound scales as $r^{2-2\beta}$, while the Gershgorin circle spark lower bound scales as $r^{1-\beta/2}$. These are asymptotically equal at $\beta=\frac{2}{3}$, and thus for any $\beta<\frac{2}{3}$, we expect the NERF bound to be asymptotically superior, and for $\beta>\frac{2}{3}$ expect the Gershgorin circles bound to be asymptotically superior. Equivalently, we expect the NERF bound to be asymptotically superior on the Naimark complements of ETFs where $N\sim r^{\beta}$ for $\beta>\frac{3}{2}$.

There are few known infinite families of ETFs where $N$ scales super-linearly with $r$. One family of examples is given by the maximal ETFs that saturate the Gerzon bound, Proposition 1.14, for which $N\sim r^{2}$, but it is perhaps the most prominent open problem in the theory of ETFs to determine in what dimensions (in both the real and complex cases) maximal ETFs exist, and it is in particular unknown if infinitely many maximal ETFs exist. Curiously, to the best of our knowledge all other known families with super-linear scaling in fact scale precisely as $N\sim r^{3/2}$. (We are aware of such families based on difference sets in finite abelian groups [8], Steiner systems corresponding to finite affine and projective planes [17], hyperovals in finite projective planes [16], and abelian generalized quadrangles [13].) On these examples, the bound we will present gives an improvement of sub-leading order on the NERF bound.

1.1.5 Steiner Equiangular Tight Frames

Finally, we present the Steiner ETF construction of [17], on which we will illustrate our results in greatest detail. This construction is based on combining two types of combinatorial objects, Steiner systems and Hadamard matrices, as defined in Section 1.1.2

Proposition 1.17** (Theorem 1 of [17]).**

Let $(S,\mathcal{B})$ be a $(2,k,v)$-Steiner system, let $b=\frac{v(v-1)}{k(k-1)}$ be the number of blocks, and let $\rho=\frac{v-1}{k-1}$ be the number of blocks containing any point. Let $\bm{H}\in\mathbb{C}^{(1+\rho)\times(1+\rho)}$ be a complex Hadamard matrix. Then, there exists an ETF of $N=\rho v$ vectors in $\mathbb{C}^{r}$ with $r=b$.

*A matrix $\bm{V}\in\mathbb{C}^{b\times\rho v}$ whose columns are the ETF vectors may be constructed as follows. Let $\bm{N}\in\{0,1\}^{v\times b}$ be the incidence matrix of the Steiner system. For each $j=1,\dots,v$, let $\bm{V}_{j}\in\mathbb{C}^{b\times\rho}$ be formed from the $j$th column of $\bm{N}^{\top}$ by replacing every entry equal to 1 with a distinct row of $\bm{H}$, and every entry equal to 0 with a row of zeros. Then, let $\bm{V}\colonequals\rho^{-1/2}[\bm{V}_{1}\cdots\bm{V}_{v}]$. If $\bm{H}$ is a real-valued Hadamard matrix, then the resulting ETF is also real. *

2 Projecting to the Perturbation Subspace

Returning to the more general setting of the introduction, let $\bm{v}_{1},\dots,\bm{v}_{N}\in\mathbb{C}^{r}$ form a UNTF, let $\bm{V}\in\mathbb{C}^{r\times N}$ have the $\bm{v}_{i}$ as its columns, and let $\bm{X}=\bm{V}^{*}\bm{V}$ be the Gram matrix. In this section, we compute the orthogonal projection operator to $\mathsf{pert}_{\widetilde{\mathscr{E}}^{N}}(\bm{X})$,

[TABLE]

The key tool is the following classical result describing perturbation subspaces for the elliptope.

Proposition 2.1** (Theorem 1(a) of [25]).**

Let $\bm{X}\in\widetilde{\mathscr{E}}^{N}$ with $\mathsf{rank}(\bm{X})=r$, and let $\bm{v}_{1},\dots,\bm{v}_{N}\in\mathbb{C}^{r}$ such that $X_{ij}=\langle\bm{v}_{i},\bm{v}_{j}\rangle$ for $i,j\in[N]$. Write $\bm{V}\in\mathbb{C}^{r\times N}$ for the matrix with the $\bm{v}_{i}$ as its columns. Then,

[TABLE]

Our formula is then the following (retaining the notation from Proposition 2.1).

Lemma 2.2**.**

Let $\bm{v}_{1},\dots,\bm{v}_{N}\in\mathbb{C}^{r}$ form a UNTF. Suppose the $\bm{v}_{i}\bm{v}_{i}^{*}$ are linearly independent, or equivalently that $|\bm{X}|^{\odot 2}$ is non-singular. Let $\bm{x}_{1},\dots,\bm{x}_{N}$ be the columns of $\bm{X}$. Then,

[TABLE]

(In particular, all ETFs except trivial ones with $r=1$ satisfy the hypotheses.)

Proof.

By Proposition 2.1 and the variational characterization (16), we have

[TABLE]

(In the final equation we use the UNTF property.) Introducing a vector of Lagrange multipliers $\bm{\gamma}\in\mathbb{R}^{N}$ for the constraints in the optimization defining $\bm{H}^{\star}$, we obtain the Lagrangian

[TABLE]

The first-order condition for optimality then implies that

[TABLE]

for some $\bm{\gamma}(\bm{A})$ such that $\bm{v}_{i}^{*}\bm{H}^{\star}(\bm{A})\bm{v}_{i}=0$ for $i\in[N]$. These constraints may be written as the system

[TABLE]

which is a linear system in $\bm{\gamma}(\bm{A})$ with matrix $|\bm{X}|^{\odot 2}$. Since this matrix is invertible by assumption, there is a unique solution

[TABLE]

The result follows by substituting into (23) and then (19).

The result we will use to obtain sparsity inequalities follows from manipulations of the fact that $\mathcal{P}$ is psd, whereby $\langle\bm{A},\mathcal{P}\bm{A}\rangle\geq 0$ for any $\bm{A}\in\mathbb{C}^{N\times N}_{\mathsf{herm}}$. Surprisingly, this fact is equivalent to the following simpler matrix inequality.

Lemma 2.3**.**

Let $\bm{v}_{1},\dots,\bm{v}_{N}\in\mathbb{C}^{r}$ form a UNTF. Suppose the $\bm{v}_{i}\bm{v}_{i}^{*}$ are linearly independent, or equivalently that $|\bm{X}|^{\odot 2}$ is non-singular. Then,

[TABLE]

or equivalently

[TABLE]

Remark**.**

*After obtaining Lemma 2.3, we discovered the reference [28], which gives a general result of the form (27) not depending on the $\bm{v}_{i}$ forming a tight frame. The proof of [28] builds a psd block matrix through the Schur product theorem and uses that its Schur complement remains psd. Our proof gives a more direct geometric argument, and, as we discuss in Section 4, may generalize to higher-degree inequalities. *

Proof.

Let us write

[TABLE]

Writing $\widehat{\bm{V}}\colonequals\sqrt{\frac{r}{N}}\bm{V}$, we have

[TABLE]

Any $\bm{H}\in\mathbb{C}^{r\times r}_{\mathsf{herm}}$ may be expressed in the form $\frac{r}{N}\widehat{\bm{V}}\bm{A}\widehat{\bm{V}}^{*}$ by taking $\bm{A}=\widehat{\bm{V}}^{*}\bm{H}\widehat{\bm{V}}$. Therefore, the inequality $\langle\bm{A},\mathcal{P}\bm{A}\rangle=\langle\bm{A},\mathcal{P}_{1}\bm{A}\rangle-\langle\bm{A},\mathcal{P}_{2}\bm{A}\rangle\geq 0$ holding for $\bm{A}\in\mathbb{C}^{N\times N}_{\mathsf{herm}}$ is equivalent to the following inequality holding for $\bm{H}\in\mathbb{C}^{r\times r}_{\mathsf{herm}}$:

[TABLE]

Moreover, since applying a unitary transformation to a UNTF produces another UNTF having the same Gram matrix $\bm{X}$, we may assume that $\bm{H}$ is diagonal and real-valued, $\bm{H}=\mathsf{diag}(\bm{\lambda})$ with $\bm{\lambda}\in\mathbb{R}^{r}$. Rewriting the resulting inequality as an inequality of quadratic forms in $\bm{\lambda}$ then gives the result. The equivalence of (26) and (27) is a general fact; see e.g. [1].

3 Sparsity Inequalities for ETFs

Our remaining results take advantage of the fact that the matrix $|\bm{X}|^{\odot 2}$ is very simple for an ETF:

[TABLE]

Moreover, by the Welch bound (our Proposition 1.13), $\alpha$ depends only on the dimension parameters $N$ and $r$:

[TABLE]

In this case, Lemma 2.3 gives the following.

Theorem 3.1**.**

Let $\bm{v}_{1},\dots,\bm{v}_{N}\in\mathbb{C}^{r}$ form an ETF. Define $\bm{R}\colonequals(|\bm{V}|^{\odot 2})(|\bm{V}|^{\odot 2})^{\top}\in\mathbb{R}^{r\times r}$, with entries $R_{k\ell}=\sum_{i=1}^{N}|(\bm{v}_{i})_{k}|^{2}|(\bm{v}_{i})_{\ell}|^{2}$. Then,

[TABLE]

Proof.

From (34) and (35), we find

[TABLE]

We have

[TABLE]

and substituting into (26) gives the result.

The next result gives the exact dimension of the subspace on which the inequality (36) is sharp on the Steiner ETFs described in Proposition 1.17 (and in greater detail in the original work [17]).

Proposition 3.2**.**

Let $\bm{v}_{1},\dots,\bm{v}_{N}\in\mathbb{R}^{r}$ be a Steiner ETF constructed from a $(2,k,v)$-Steiner system and a Hadamard matrix of suitable size. Let $\bm{R}\in\mathbb{R}^{r\times r}$ have entries $R_{k\ell}=\sum_{i=1}^{N}|(\bm{v}_{i})_{k}|^{2}|(\bm{v}_{i})_{\ell}|^{2}$. Then,

[TABLE]

*When the Steiner system is a finite projective plane, then the inequality (36) is an equality of matrices. *

Proof.

Let $b=\frac{v(v-1)}{k(k-1)}$ be the number of blocks in the underlying Steiner system and $\rho=\frac{v-1}{k-1}$ be the number of blocks in which each point lies. Then, per the construction of [17] described in Proposition 1.17, $r=b$ and $N=v(1+\rho)$.

We first compute $\bm{R}$: letting $\bm{N}\in\mathbb{R}^{v\times b}$ be the incidence matrix of points and blocks of the Steiner system, we find

[TABLE]

where $\bm{A}_{G}$ is the adjacency matrix of the block intersection graph $G$ of the Steiner system. $G$ is a strongly regular graph [20], admitting a spectral expansion

[TABLE]

where $\widehat{\bm{1}}_{b}=\frac{1}{\sqrt{b}}\bm{1}_{b}$, $U_{\pm}$ are eigenspaces orthogonal to one another and to the vector $\bm{1}$ and satisfying $U_{+}\oplus U_{-}\oplus\bm{1}=\mathbb{R}^{b}$, and $\bm{P}_{U_{\pm}}$ are the projectors onto these subspaces. The corresponding dimensions are

[TABLE]

We thus obtain the spectral expansion of $\bm{R}$,

[TABLE]

Some algebraic manipulations show that the following identities hold between the eigenvalues of the right-hand side of (36) and those of $\bm{R}$:

[TABLE]

and therefore in fact

[TABLE]

The first part of the result then follows from the dimension formula (42). For the special case of finite projective planes, it suffices to note that in this case $r=b=v$, which is easily verified from the formulae in Proposition 1.11

Finally, to illustrate how some more concrete results may be obtained from Theorem 3.1, we give corollaries controlling ETF sparsity, spark, and the overlap of rows of the synthesis matrix $\bm{V}$. Recall that the spark is defined as

[TABLE]

The natural dual measure of sparsity, sometimes called cospark, is

[TABLE]

which gives control of the sparsity of the entire matrix $\bm{V}$ by controlling each row.

Corollary 3.3**.**

Let $\bm{v}_{1},\dots,\bm{v}_{N}\in\mathbb{C}^{r}$ form an ETF, and let $\bm{V}\in\mathbb{C}^{r\times N}$ have the $\bm{v}_{i}$ as its columns. Then,

[TABLE]

Proof.

Let $\bm{y}\in\mathbb{C}^{r}$ with $\|\bm{y}\|_{2}=1$. Let $\bm{U}\in\mathbb{C}^{r\times r}$ be a unitary matrix whose first column is $\bm{y}$. We apply (36) to the ETF with synthesis matrix $\bm{U}\bm{V}$, whose first row is $\bm{y}^{*}\bm{V}$. Comparing the entries in the upper left corners of the matrices on either side of the inequality, we find

[TABLE]

Writing $\bm{x}=\bm{V}^{*}\bm{y}$, note that in general, by the Cauchy-Schwarz inequality,

[TABLE]

In our case, $\|\bm{x}\|_{2}^{2}=\bm{y}^{*}\bm{V}\bm{V}^{*}\bm{y}=\frac{N}{r}$, and thus

[TABLE]

This gives the sparsity result, and the spark result follows from the same argument applied to the Naimark complement by (14).

We compare this bound to the NERF and Gershgorin bounds of Proposition 1.15 and Proposition 1.16 respectively in Table 1, and find that on the Naimark complements of infinite families of ETFs with dimensions scaling as $N\sim r^{3/2}$, our bound gives an improvement of sub-leading order on the NERF bound. On the other hand, both our bound and the NERF bound are incomparable in general to the Gershgorin circle bound on such ETFs, sometimes being superior and sometimes inferior depending on the specific construction.

The final corollary we mention shows that the overlap between sparsity patterns of distinct rows of an ETF in fact has a certain “typical” value for a given pair of dimensions $r$ and $N$, from which its possible deviations are bounded.

Corollary 3.4**.**

Let $\bm{v}_{1},\dots,\bm{v}_{N}\in\mathbb{C}^{r}$ form an ETF, and let $\bm{V}\in\mathbb{R}^{r\times N}$ have the $\bm{v}_{i}$ as its columns. Let $\bm{a},\bm{b}\in\mathsf{row}(\bm{V})$ with $\langle\bm{a},\bm{b}\rangle=0$ and $\|\bm{a}\|_{2}^{2}=\|\bm{b}\|_{2}^{2}=\frac{N}{r}$ (for instance, two distinct rows of $\bm{V}$). Let $D\colonequals\frac{N}{r^{2}}(1+\frac{(r-1)^{2}}{N-1})$ and $E\colonequals\frac{\frac{N}{r}-1}{r(1-\frac{1}{N})}$. Then, $D\geq\|\bm{a}\|_{4}^{4}$ and $D\geq\|\bm{b}\|_{4}^{4}$, and

[TABLE]

Proof.

Note that $D$ is equal to the diagonal entries and $E$ to the off-diagonal entries of the right-hand side of (36). Thus, $D\geq\|\bm{a}\|_{4}^{4}$ and $D\geq\|\bm{b}\|_{4}^{4}$ follows by the previous argument of Corollary 3.3. As in that argument, we may without loss of generality assume $\bm{a}$ and $\bm{b}$ occur as the first two rows of $\bm{V}$. Then, the relation between the upper left $2\times 2$ minors of either side of (36) is

[TABLE]

Taking the determinant of the difference of the right- and left-hand sides of the above then gives the result.

4 Open Problems

Returning to the more general family of relaxations of the cut polytope described in Definition 1.2, the main question we propose for investigation is the following.

Question 4.1**.**

*Given the Gram matrix of a real ETF of $N$ vectors in $\mathbb{R}^{r}$, what is the largest $d$ for which it belongs to $\mathscr{E}_{d}^{N}$? Does the answer depend only on $N$ and $r$? *

More specifically, we are interested in the details of the construction that would underlie such a result.

Question 4.2**.**

*When $\bm{X}$ is the Gram matrix of a real ETF and $\bm{X}\in\mathscr{E}_{d}^{N}$ with a “witness” $\bm{Y}$ per Definition 1.2, is there a tractable description of the eigenspaces of $\bm{Y}$? *

If this is the case, then we may hope to imitate the present approach: compute projectors to the analogous subspaces for complex ETFs, write down the positivity relation for these operators, and derive polynomial inequalities in the ETF entries. It remains to be seen, however, whether such inequalities could still be interpreted as giving information about sparsity of ETFs, or whether they would control new quantities for higher degrees.