Rank optimality for the Burer-Monteiro factorization

TL;DR

This paper investigates the conditions under which the Burer-Monteiro factorization reliably finds global solutions in large-scale semidefinite programs, showing that certain size thresholds are essentially tight for guaranteeing optimality.

Contribution

The paper proves that the known size threshold for the Burer-Monteiro factorization to avoid spurious critical points is essentially tight, especially for rank-1 solutions.

Findings

Second-order critical points are not generically optimal below the size threshold.

The size threshold for guaranteed global optimality is tight for rank-1 solutions.

The results delineate the limitations of the Burer-Monteiro approach in low-rank regimes.

Abstract

When solving large scale semidefinite programs that admit a low-rank solution, an efficient heuristic is the Burer-Monteiro factorization: instead of optimizing over the full matrix, one optimizes over its low-rank factors. This reduces the number of variables to optimize, but destroys the convexity of the problem, thus possibly introducing spurious second-order critical points. The article [Boumal, Voroninski, and Bandeira, 2018] shows that when the size of the factors is of the order of the square root of the number of linear constraints, this does not happen: for almost any cost matrix, second-order critical points are global solutions. In this article, we show that this result is essentially tight: for smaller values of the size, second-order critical points are not generically optimal, even when the global solution is rank 1.

Equations

$$\mathrm{Trace}(CX)$$

$$\mathcal{A}(X)=b,$$

$$X⪰0,$$

$$\mathrm{Trace}(CU{U}^T)$$

$$\mathcal{A}(U{U}^T)=b,$$

$$U\in {\mathbb{R}}^{n\times p}.$$

$$\frac{p(p+1)}{2}>m,$$

$$\frac{p(p+1)}{2}+p>m.$$

$$\frac{p(p+1)}{2}+p{r}_{\ast }\le m,$$

$$\forall M_1,M_2\in {\mathbb{R}}^{n_1\times n_2},⟨M_1,M_2⟩\stackrel{def}{=}\mathrm{Tr}(M_1^T{M}_2).$$

$$⟨C,X⟩$$

$$\mathcal{A}(X)=b,$$

$$X⪰0.$$

$$\mathcal{C}=\{X\in {\mathbb{S}}^{n\times n},\mathcal{A}(X)=b,X⪰0\}.$$

$$⟨C,V{V}^T⟩$$

$$\mathcal{A}(V{V}^T)=b,$$

$$V\in {\mathbb{R}}^{n\times p}.$$

$${\mathcal{M}}_p=\{V\in {\mathbb{R}}^{n\times p},\mathcal{A}(V{V}^T)=b\}.$$

$$\dot{V}\in {\mathbb{R}}^{n\times p}\to \mathcal{A}(V{\dot{V}}^T+\dot{V}{V}^T)\in {\mathbb{R}}^m.$$

$$T_V{\mathcal{M}}_p=\{\dot{V}\in {\mathbb{R}}^{n\times p},\mathcal{A}(V{\dot{V}}^T+\dot{V}{V}^T)=0\}.$$

$$\mathrm{Hess}{f}_C(V)\cdot (\dot{V},\dot{V})=2⟨C_2,\dot{V}{\dot{V}}^T⟩,$$

$$\forall \dot{V}\in T_V{\mathcal{M}}_p,⟨C_2,\dot{V}{\dot{V}}^T⟩\ge 0,$$

$$\mathrm{rank}(\mathrm{Hess}{f}_C(V))$$

$$=\dim ({\mathcal{M}}_p)-\frac{p(p-1)}{2}.$$

$$T_V{\mathcal{M}}_p\cap \{\dot{V}\in {\mathbb{R}}^{n\times p},\mathrm{Range}(\dot{V})\subset \mathrm{Range}(X_0)+\mathrm{Range}(V)\}$$

$$\frac{p(p+1)}{2}>m,\left(⟺p>\lfloor \sqrt{2m+\frac{1}{4}}-\frac{1}{2}\rfloor \right)$$

$$\frac{p(p+1)}{2}+pr\le m.$$

$$⟨C,X⟩$$

$$\mathrm{diag}(X)=1,$$

$$X⪰0.$$

$$⟨C,X⟩$$

$$X\in {\mathbb{S}}^{Sd\times Sd},$$

$${\mathrm{Block}}_s(X)=I_d,\forall s=1,\dots ,S,$$

$$X⪰0,$$

$$⟨C,X⟩$$

$$X\in {\mathbb{S}}^{Sd\times Sd},$$

$$\mathcal{A}(X)=b,$$

$$X⪰0,$$

$$\frac{p(p+1)}{2}+p>\frac{Sd(d+1)}{2},$$

Full text

1 Introduction

We consider a semidefinite program:

[TABLE]

where the variable $X$ and the fixed matrix $C$ are symmetric, of size $n\times n$, and $\mathcal{A}$ is a linear operator capturing $m$ equality constraints.

Various iterative algorithms have been developed to solve such a problem at a given precision level, but tend to be computationally demanding. For example, in full generality, each iteration may cost $O((m+n)mn^{2})$ arithmetic operations with an interior-point solver [Borchers and Young, 2007, Page 357] (assuming $m\leq\mathrm{dim}(\mathrm{Sym}(n))=\frac{n(n+1)}{2}$), and $O((m+n)n^{2})$ with first-order techniques applied to a smoothed version of the problem [Nesterov, 2005, Section 3].

Improvements are possible if $\mathcal{A}$ has some structure that can be exploited, but they often do not suffice to make large-scale semidefinite programs computationally easy. Another property can then be used: semidefinite programs tend to have a low-rank minimizer (in many applications, there is one with rank $O(1)$, and, in any case, always one with rank $\sim\sqrt{2m}$ [Pataki, 1998, Theorem 2.1]). Low-rank matrices can be stored and manipulated in a much more efficient way than full-rank ones, which allows for less computationally demanding algorithms.

Frank-Wolfe methods, in particular, take advantage of this [Jaggi, 2013; Laue, 2012; Yurtsever, Udell, Tropp, and Cevher, 2017]. Here, we are interested in another approach, the Burer-Monteiro factorization [Burer and Monteiro, 2005]. Its principle is that a semidefinite matrix with rank $p\ll n$ can be factorized as $X=UU^{T},$ with $U\in\mathbb{R}^{n\times p}$. Assuming that a low-rank solution $X_{opt}$ exists, if $p\geq\mathrm{rank}(X_{opt})$, Problem (SDP) is then equivalent to

[TABLE]

Now the unknown $U$ has $np$ coordinates, fewer than the $n^{2}$ coordinates of $X$. Consequently, we can run on Problem (Factorized SDP) local optimization algorithms that would be too slow on Problem (SDP). The caveat is that, since the factorized problem is not convex, they are not guaranteed to find a global minimizer, at best a second-order critical point. Nevertheless, they work extremely well in many applications. Typically, as soon as $p$ is slightly larger than $\mathrm{rank}(X_{opt})$, local optimization algorithms seem to globally solve Problem (Factorized SDP). Numerical examples where this phenomenon occurs can be found in [Burer and Monteiro, 2003], [Journée, Bach, Absil, and Sepulchre, 2010, Section 5], [Boumal, 2015, Section 5] or [Rosen, Carlone, Bandeira, and Leonard, 2016, Section 5].

The article [Bandeira, Boumal, and Voroninski, 2016a] gives a rigorous explanation of this behavior for instances of (Factorized SDP) coming from $\mathbb{Z}_{2}$-synchronization and community detection. It notably establishes, in particular statistical regimes where Problem (SDP) has a rank-$1$ solution, that all second-order critical points of Problem (Factorized SDP) with $p=2$ are global minimizers. Hence, suitable local optimization algorithms globally solve Problem (Factorized SDP). Similarly, [Ge, Lee, and Ma, 2016; Sun, Qu, and Wright, 2017; Li, Zhu, and Tang, 2018] show, in related settings, that all second-order critical points of the Burer-Monteiro factorization are the optimal solution as soon as $p\geq\mathrm{rank}(X_{opt})$.

But these works apply in very specific settings only. They provide no general theory on when local optimization algorithms solve Problem (Factorized SDP). With no restrictive assumptions, essentially the only result is [Boumal, Voroninski, and Bandeira, 2018]. Building on [Burer and Monteiro, 2005] and [Boumal, 2015], it shows that, under simple hypotheses, all second-order critical points of Problem (Factorized SDP) are global minimizers, for almost any matrix $C$, as soon as

[TABLE]

that is $p>\lfloor\sqrt{2m+1/4}-1/2\rfloor$. Extensions can be found in [Pumir, Jelassi, and Boumal, 2018; Bhojanapalli, Boumal, Jain, and Netrapalli, 2018].

As a result there is a gap in the literature: in all the concrete settings that could be studied, all second-order critical points of Problem (Factorized SDP) are global minimizers as soon as $p\gtrsim\mathrm{rank}(X_{opt})$, in line with numerical experiments, but in the general case, the only guarantees at our disposal state that we need $p$ to be at least as large as $\sim\sqrt{2m}$. In many applications, $\mathrm{rank}(X_{opt})=O(1)$ while $m=O(n)$, hence these two estimates are far apart, making a huge difference on the computational cost of certifiable algorithms.

The natural question is, ”can the gap be reduced?” In this article, we negatively answer this question, and show that Inequality (1.1) is essentially optimal. It can be slightly improved, to

[TABLE]

This is Theorem 1. But Theorem 2 (our main result) states that, under reasonable assumptions on $\mathcal{A},b$, if $p$ is such that

[TABLE]

where $r_{*}=\min\{\mathrm{rank}(X),X\succeq 0,\mathcal{A}(X)=b\}$, there exists a set of cost matrices $C$ with non-zero Lebesgue measure on which Problem (SDP) admits a global minimizer with rank $r_{*}$, but Problem (Factorized SDP) has second-order critical points which are not global minimizers. In particular, if $r_{*}=1$ (as is the case in MaxCut relaxations, for instance), Inequality (1.2) is exactly optimal. Therefore without specific assumptions on $C$, when running a local optimization algorithm on Problem (Factorized SDP) with $p$ smaller than $\sim\sqrt{2m}$, we cannot be sure not to run into a spurious second-order critical point, even if there exists a global minimizer with rank $O(1)$.

Regarding the organization of this article, Section 2 contains basic definitions (Subsection 2.1) and properties (Subsection 2.2), defines and discusses an important assumption for our main result (Subsection 2.3). Section 3 presents the main results: Theorems 1 and 2 are respectively stated in Subsections 3.1 and 3.2. Subsection 3.3 provides examples. The other sections contain the proofs: Theorem 1 is proved in Section 4, and Theorem 2 in Section 5.

1.1 Notation

For any $p,q\in\mathbb{N}^{*}$, we denote by $I_{p}$ the $p\times p$ identity matrix, and by $0_{p,q}$ the zero $p\times q$ matrix. For any $p\in\mathbb{N}^{*}$, we denote by $\mathbb{S}^{p\times p}$ the set of real symmetric $p\times p$ matrices, by $\operatorname{Anti}(p)$ the set of antisymmetric $p\times p$ matrices, and by $O(p)$ the set of orthogonal $p\times p$ matrices. For any $n_{1},n_{2}$, we equip $\mathbb{R}^{n_{1}\times n_{2}}$, the set of $n_{1}\times n_{2}$ matrices, with the usual scalar product:

[TABLE]

The same formula also defines a scalar product on $\mathbb{S}^{p\times p}$, for any $p\in\mathbb{N}^{*}$. In both cases, the associated norm is the Frobenius norm, which we denote by $||.||_{F}$. For any $p\in\mathbb{N}^{*}$, we define $\mathrm{diag}:\mathbb{R}^{p\times p}\to\mathbb{R}^{p}$ as the operator which associates to a matrix the vector of its diagonal elements.

For any element $x$ of a metric space, and any positive $\epsilon$, we denote $B(x,\epsilon)$ the open ball with radius $\epsilon$, and $\overline{B}(x,\epsilon)$ the closed ball. When $\mathcal{M}$ is a manifold, and $x$ an element of $\mathcal{M}$, we denote by $T_{x}\mathcal{M}$ the tangent space of $\mathcal{M}$ at $x$.

2 Preliminaries

2.1 Definitions

We consider a problem of the following form:

[TABLE]

Here, $\mathcal{A}:\mathbb{S}^{n\times n}\to\mathbb{R}^{m}$ is a fixed linear map, $b$ a fixed element of $\mathbb{R}^{m}$, and $C$ an element of $\mathbb{S}^{n\times n}$, which is called the cost matrix.

We denote by $\mathcal{C}$ the set of feasible points for this problem:

[TABLE]

As explained in the introduction, if we assume that Problem (SDP) has an optimal solution $X_{opt}$ with rank $r$, and fix some $p\geq r$, it is equivalent to its rank $p$ Burer-Monteiro factorization:

[TABLE]

We denote by $\mathcal{M}_{p}$ the set of feasible points for the factorized problem:

[TABLE]

It is invariant under multiplication by elements of $O(p)$. We assume that it is sufficiently regular so that we can apply smooth optimization algorithms to Problem (Factorized SDP). More precisely, all our results require that $(\mathcal{A},b)$ is $p$-regular:

Definition 1**.**

For some $p\in\mathbb{N}^{*}$, $(\mathcal{A},b)$ is said to be $p$-regular if, for all $V\in\mathcal{M}_{p}$, the following linear map is surjective:

[TABLE]

This assumption is of the same style as [Boumal, Voroninski, and Bandeira, 2018, Assumption 1.1]. It notably guarantees [Absil, Mahony, and Sepulchre, 2009, Proposition 3.3.3] that $\mathcal{M}_{p}$ is a submanifold of $\mathbb{R}^{n\times p}$, with dimension $\dim(\mathcal{M}_{p})=np-m$, whose tangent space at any point $V$ is

[TABLE]

The scalar product of $\mathbb{R}^{n\times p}$ defines a metric on the manifold $\mathcal{M}_{p}$, which we then view as a Riemannian manifold. Many algorithms exist for attempting to minimize a smooth function on a Riemannian manifold; a classical reference on this topic is [Absil, Mahony, and Sepulchre, 2009].

However, they are a priori not guaranteed to find a global minimizer of Problem (Factorized SDP), but only an (approximate) first or second-order critical point of the cost function $V\in\mathcal{M}_{p}\to\left\langle C,VV^{T}\right\rangle$ [Boumal, Absil, and Cartis, 2016]. These points are defined as follows:

Definition 2**.**

Let $\mathcal{N}$ be a Riemannian manifold, and $f:\mathcal{N}\to\mathbb{R}$ a smooth function. We denote $\nabla$ and $\mathrm{Hess}$ its gradient and Hessian with respect to the manifold.

For any $x_{0}\in\mathcal{N}$, we say that $x_{0}$ is a first-order critical point of $f$ if $\nabla f(x_{0})=0$ and a second-order critical point of $f$ if $\nabla f(x_{0})=0$ and $\mathrm{Hess}f(x_{0})\succeq 0.$

The goal of this article is to study for which values of $p$ the set of second-order critical points coincides with the set of global minimizers of Problem (Factorized SDP).

We note that there are pairs $(\mathcal{A},b)$ which are not $p$-regular, regardless of the value of $p$ (an example is when $0_{n,n}\in\mathcal{C}$). This setting is significantly different from the one that we consider in this article: $\mathcal{M}_{p}$ may then have singularities, and classical Riemannian tools are a priori not applicable to Problem (Factorized SDP).

2.2 Basic properties

It is convenient to be able to describe the solutions of Problem (SDP) in terms of Karush-Kuhn-Tucker conditions. This is a priori possible only if strong duality holds but, fortunately for us, strong duality always holds when $(\mathcal{A},b)$ is $p$-regular for some $p$, yielding the following proposition (whose proof is in Appendix A.1).

Proposition 1**.**

We assume that there exists $p\in\mathbb{N}^{*}$ such that $(\mathcal{A},b)$ is $p$-regular and $\mathcal{M}_{p}\neq\emptyset$. Then a matrix $X_{0}\in\mathcal{C}$ is a solution of Problem (SDP) if and only if there exist $g_{1}\in\mathbb{R}^{m},C_{1}\in\mathbb{S}^{n\times n}$ such that

• •

$C=\mathcal{A}^{*}(g_{1})+C_{1}$;

• •

$C_{1}\succeq 0$;

• •

$C_{1}X_{0}=0$.

When, $g_{1},C_{1}$ satisfy $\mathrm{rank}(C_{1})=n-\mathrm{rank}(X_{0})$ in addition to the above three conditions, we say that strict complementary slackness holds. The following proposition (whose proof is in Appendix A.2) states that, under an additional condition on $X_{0}$, it implies that the solution of Problem (SDP) is unique.

Proposition 2**.**

If strict complementary slackness holds and $X_{0}$ is an extremal point of $\mathcal{C}$, then $X_{0}$ is the unique solution of Problem (SDP).

The next proposition characterizes, in a similar way as Proposition 1, the first-order critical points of Problem (Factorized SDP). Its proof is in Appendix A.3.

Proposition 3**.**

We assume that $(\mathcal{A},b)$ is $p$-regular for some $p\in\mathbb{N}^{*}$. A matrix $V\in\mathcal{M}_{p}$ is a first-order critical point of Problem (Factorized SDP) if and only if there exist $g_{2}\in\mathbb{R}^{m},C_{2}\in\mathbb{S}^{n\times n}$ such that

• •

$C=\mathcal{A}^{*}(g_{2})+C_{2}$;

• •

$C_{2}V=0$.

When it exists, the pair $(g_{2},C_{2})$ is unique.

Finally, we also provide a reformulation of second-order criticality; the proof is in Appendix A.4.

Proposition 4**.**

We assume that $(\mathcal{A},b)$ is $p$-regular for some $p\in\mathbb{N}^{*}$. Let $V\in\mathcal{M}_{p}$ be a first-order critical point of Problem (Factorized SDP), whose cost function we denote $f_{C}$. For any $\dot{V}\in T_{V}\mathcal{M}_{p}$,

[TABLE]

with $C_{2}$ defined as in Proposition 3. Thus, $V$ is second-order critical if and only if

[TABLE]

Using the notation of the previous proposition, we observe that, since $f_{C}$ is invariant under right multiplication by elements of $O(p)$, $\mathrm{Hess}f_{C}(V)\cdot(\dot{V},\dot{V})=0$ for any $\dot{V}$ tangent to the orbit of $V$ under the action of $O(p)$, that is $\dot{V}=VA$ for some $A\in\operatorname{Anti}(p)$. This motivates the following definition.

Definition 3**.**

A second-order critical point $V$ of Problem (Factorized SDP) is non-degenerate if, in Equation (2.1), the equality is attained exactly for matrices $\dot{V}$ of the form $\dot{V}=VA,A\in\operatorname{Anti}(p)$.

Remark 1**.**

Equivalently, a second-order critical point is non-degenerate if

[TABLE]

2.3 Definition of “minimally secant”

The following technical property is needed for our main theorem.

Definition 4**.**

Let $p\in\mathbb{N}^{*}$ be such that $(\mathcal{A},b)$ is $p$-regular. Let $r$ be in $\mathbb{N}^{*}$.

Let $X_{0}$ be a rank $r$ element of $\mathcal{C}$ and $V$ be in $\mathcal{M}_{p}$. We say that $\mathcal{M}_{p}$ is $X_{0}$-minimally secant at $V$ if the following three conditions hold:

$\mathrm{rank}(V)=p$; 2. 2.

$\mathrm{Range}(X_{0})\cap\mathrm{Range}(V)=\{0\}$; 3. 3.

for any $\dot{V}\in T_{V}\mathcal{M}_{p}$, if $\mathrm{Range}(\dot{V})\subset\mathrm{Range}(X_{0})+\mathrm{Range}(V)$, then $\dot{V}=VA$ for some $A\in\operatorname{Anti}(p)$.

We observe that, for any $V\in\mathcal{M}_{p}$, the intersection

[TABLE]

necessarily contains $\{VA,A\in\operatorname{Anti}(p)\}$. Therefore, the third property in the above definition amounts to requiring that the intersection is “as small as possible” (hence the name “minimally secant”).

Our main result, Theorem 2, contains the assumption that there exists $X_{0},V$ such that $\mathcal{M}_{p}$ is $X_{0}$-minimally secant at $V$. This assumption is crucial for the proof: to show the existence of at least one cost matrix $C$ for which a spurious second-order critical point exists, our strategy is to fix $X_{0}$ and $V$, and construct $C$ for which a second-order critical point exists and is precisely $V$, while the global minimizer corresponds to $X_{0}$. To ensure first and second-order criticality for $V$, and global optimality for $X_{0}$, we essentially need that the quadratic form defined by $C$ satisfies some properties when restricted to $\mathrm{Range}(X_{0})$, some other properties on $\mathrm{Range}(V)$, and still some other ones on $\mathrm{Range}(\dot{V})$ for $\dot{V}\in T_{V}\mathcal{M}_{p}$. In order for these properties to be compatible with each other, the spaces $\mathrm{Range}(X_{0}),\mathrm{Range}(V),\mathrm{Range}(\dot{V})$ must “not intersect too much”. The formal content behind “not intersecting too much” is precisely Definition 4.

However, as explained in Appendix B, when $\frac{p(p+1)}{2}+pr\leq m$, we expect such $X_{0},V$ to almost always exist. This is notably the case for MaxCut problems (see Paragraph 3.3.1 and Appendix F.1), and also for Orthogonal-Cut (Paragraph 3.3.2) and optimization over a product of spheres (Paragraph 3.3.3).

3 Main results

3.1 Regime where critical points are global minimizers

As previously stated, most smooth optimization algorithms, applied to Problem (Factorized SDP), are only guaranteed to find a critical point of this problem, and not a global minimizer. Fortunately, [Boumal, Voroninski, and Bandeira, 2018] shows that, when $p$ is large enough, second-order critical points are always global minimizers, for almost all cost matrices $C$. Therefore, algorithms able to find second-order critical points (for instance, the trust-region method) actually solve Problem (Factorized SDP) to optimality, provided that $C$ is “generic”. A restated version of the theorem in [Boumal, Voroninski, and Bandeira, 2018], under minor modifications, is the following:

Theorem**.**

[Boumal, Voroninski, and Bandeira, 2018, Theorem 1.4]* Let $p\in\mathbb{N}^{*}$ be fixed. We assume that*

The set $\mathcal{C}$ of feasible points for Problem (SDP) is compact; 2. 2.

$(\mathcal{A},b)$* is $p$-regular.*

If

[TABLE]

then, for almost all cost matrices $C\in\mathbb{S}^{n\times n}$, if $V\in\mathcal{M}_{p}$ is a second-order critical point of Problem (Factorized SDP), then

• •

$V$* is a global minimizer of Problem (Factorized SDP);*

• •

$X=VV^{T}$* is a global minimizer of Problem (SDP).*

It is natural to ask whether Condition (3.1) is optimal, or whether the same guarantees hold for smaller ranks $p$, allowing further reductions in the computational complexity of solving Problem (Factorized SDP). Our first result is that Condition (3.1) can be slightly relaxed.

Theorem 1**.**

Let $p\in\mathbb{N}^{*}$ be fixed. We assume that

The set $\mathcal{C}$ of feasible points for Problem (SDP) is compact. 2. 2.

$(\mathcal{A},b)$* is $p$-regular;*

If $\frac{p(p+1)}{2}+p>m,$ then the same conclusion holds as in the previous theorem.

The proof of Theorem 1 is in Section 4.

3.2 Regime where there may be bad critical points

We can now address our main question: How optimal is the result of the previous section? When Problem (SDP) has a unique global minimizer, with rank $r$ of the order of $\sqrt{2m}$, the result cannot be significantly improved: $p\geq r$ is a necessary condition for Problems (SDP) and (Factorized SDP) to have the same minimum. However, as said in the introduction, Problem (SDP) often admits a solution with rank $r\ll\sqrt{2m}$, and the Burer-Monteiro factorization is numerically observed to work when $p=O(r)$.

Our main theorem however states that, even if we assume $r\ll\sqrt{2m}$, our previous result is essentially not improvable without additional hypotheses on $C$: Under reasonable assumptions on $(\mathcal{A},b)$, if $\frac{p(p+1)}{2}+pr\leq m$, there is a set of cost matrices with non-zero Lebesgue measure for which Problem (SDP) has a rank $r$ optimal solution, but Problem (Factorized SDP) has a non-optimal second-order critical point. In particular, for $r=1$, the inequality $\frac{p(p+1)}{2}+p>m$ in Theorem 1 is exactly optimal.

Theorem 2**.**

Let $r\in\mathbb{N}^{*}$ be fixed. Let $p\geq r$ be such that

[TABLE]

We make the following hypotheses:

$\mathcal{C}$* has at least one extreme point with rank $r$, denoted by $X_{0}$;* 2. 2.

$(\mathcal{A},b)$* is $p$-regular;* 3. 3.

There exists $V\in\mathcal{M}_{p}$ such that $\mathcal{M}_{p}$ is $X_{0}$-minimally secant at $V$.

Then there exists a subset $\mathcal{E}_{bad}$ of $\mathbb{S}^{n\times n}$ with non-zero Lebesgue measure such that, for any cost matrix $C\in\mathcal{E}_{bad}$,

• •

Problem (SDP) has a unique global minimizer, which has rank $r$.

• •

Problem (Factorized SDP) has at least one second-order critical point that is not a global minimizer.

The proof of Theorem 2 is in Section 5.

Remark 2**.**

Theorem 2 stays valid if one replaces “second-order critical point” with “local minimizer”. Indeed, it turns out that the second-order critical points constructed in our proof are non-degenerate, and therefore, local minimizers.

Remark 3**.**

The inequalities $\frac{p(p+1)}{2}+p>m$ and $\frac{p(p+1)}{2}+pr\leq m$ in Theorems 1 and 2 are exactly complementary when $r=1$. When $r\geq 2$, there is a small gap between them. Appendix C shows through an example that there are settings where $\frac{p(p+1)}{2}+p\leq m<\frac{p(p+1)}{2}+pr$ and the conclusions of Theorem 2 still hold, but we do not know whether it is always the case.

3.3 Examples

3.3.1 MaxCut

In this subsection, we apply our results to the most famous instance of a problem with the form (SDP), the MaxCut relaxation:

[TABLE]

This problem is a relaxation of the “maximum cut” problem from graph theory [Delorme and Poljak, 1993; Poljak and Rendl, 1995], made famous by the work [Goemans and Williamson, 1995]. It also appears in phase retrieval [Waldspurger, d’Aspremont, and Mallat, 2015] and $\mathbb{Z}/2\mathbb{Z}$ synchronization [Abbe, Bandeira, and Hall, 2016; Bandeira, Boumal, and Voroninski, 2016a] (in which cases its global optimizer is known, both theoretically and numerically, to often have very low rank, typically $1$).

Theorems 1 and 2 exactly describe when its Burer-Monteiro factorization has no non-optimal second-order critical point for almost any cost matrix, even if we assume that the global minimizer has rank $1$.

Corollary 1**.**

If $p\in\mathbb{N}$ is such that $\frac{p(p+1)}{2}+p>n$ then, for almost any cost matrix $C$, all second-order critical points of the Burer-Monteiro factorization of Problem (SDP-Maxcut) are globally optimal.

On the other hand, for any $p$ such that $\frac{p(p+1)}{2}+p\leq n$, the set of cost matrices admits a subset with non-zero Lebesgue measure on which

• •

Problem (SDP-Maxcut) has a unique global minimizer, which has rank $1$;

• •

Its Burer-Monteiro factorization with rank $p$ has at least one non-optimal second-order critical point.

This result is proved in Appendix F.1.

3.3.2 Orthogonal-Cut

We now consider a generalization of MaxCut, coined Orthogonal-Cut in [Bandeira, Kennedy, and Singer, 2016b]:

[TABLE]

where $d,S$ belong to $\mathbb{N}^{*}$ (with, typically, $d=1,2$ or $3$) and, for any $M\in\mathbb{R}^{Sd\times Sd}$, $s\leq S$, $\mathrm{Block}_{s}(M)$ is the $s$-th diagonal $d\times d$ block of $M$. Observe that this is exactly Problem (SDP-Maxcut) when $d=1$.

Problem (SDP-Orthogonal-Cut) is a natural relaxation of some optimization problems on $O(d)^{S}$. It notably has applications in molecular imaging [Wang, Singer, and Wen, 2013], sensor network localization [Cucuringu, Lipman, and Singer, 2012] and ranking [Cucuringu, 2016]. For some theoretical analysis of this semidefinite problem, including conditions under which it admits a low-rank global minimizer, the reader can refer, not only to [Bandeira, Kennedy, and Singer, 2016b], but to [Chaudhury, Khoo, and Singer, 2015], [Rosen, Carlone, Bandeira, and Leonard, 2016] or [Eriksson, Olsson, Kahl, and Chin, 2018], as well.

Problem (SDP-Orthogonal-Cut) is exactly equivalent to

[TABLE]

with $\mathcal{A}:X\in\mathbb{S}^{Sd\times Sd}\to(T_{sup}(\mathrm{Block}_{1}(X)),\dots,T_{sup}(\mathrm{Block}_{S}(X)))\in\mathbb{R}^{Sd(d+1)/2}$, and $b=(T_{sup}(I_{d}),\dots,T_{sup}(I_{d}))\in\mathbb{R}^{Sd(d+1)/2}$, where $T_{sup}:\mathbb{R}^{d\times d}\to\mathbb{R}^{d(d+1)/2}$ is the operator that extracts the $\frac{d(d+1)}{2}$ coefficients of the upper triangular part of a matrix.

With these definitions, $(\mathcal{A},b)$ is $p$-regular for any $p\in\mathbb{N}$. In particular, $\mathcal{M}_{p}$ is a manifold (non-empty if and only if $p\geq d$).

Corollary 2**.**

Let us assume that $d=1,2$ or $3$. If

[TABLE]

then, for almost any cost matrix $C$, all second-order critical points of the Burer-Monteiro factorization of Problem (SDP-Orthogonal-Cut) are globally optimal.

On the other hand, for any $p\geq d$ such that

[TABLE]

there is a set of cost matrices, with non-zero Lebesgue measure, on which

• •

Problem (SDP-Orthogonal-Cut) has a unique minimizer, whose rank is $d$;

• •

Its Burer-Monteiro factorization with rank $p$ has at least one non-optimal second-order critical point.

The proof is in Appendix F.2.

3.3.3 Optimization over a product of spheres

As a final example, let us consider the problem

[TABLE]

where $S,d_{1},\dots,d_{S}$ belong to $\mathbb{N}^{*}$, and $D=d_{1}+\dots+d_{S}$. This is the natural semidefinite relaxation of problems that consist in minimizing a degree $2$ polynomial function on the product of spheres $S^{d_{1}}\times\dots\times S^{d_{S}}$.

Problem (SDP-Product) encompasses several important particular cases: when $d_{1}=\dots=d_{S}=1$, we recover Problem (SDP-Maxcut). When $d_{1}=\dots=d_{S}=2$, it is equivalent to a complex version of (SDP-Maxcut) (for matrices $C$ of a particular form). When $S=2$ and $d_{2}=1$, it is the relaxation of a standard trust-region subproblem [Boumal, Voroninski, and Bandeira, 2018, Subsection 5.2]. For general values of $d_{1},\dots,d_{S}$, it is a simplification of the relaxation of optimization problems over an intersection of ellipsoids, which appear in trust-region algorithms for constrained problems [Celis, 1985].

Corollary 3**.**

If $p\in\mathbb{N}$ is such that

[TABLE]

then, for almost any cost matrix $C$, all second-order critical points of the Burer-Monteiro factorization of Problem (SDP-Product) are globally optimal.

On the other hand, for any $p\in\mathbb{N}^{*}$ such that

[TABLE]

the set of cost matrices admits a subset with non-zero Lebesgue measure on which

• •

Problem (SDP-Product) has a unique global optimum, which has rank $1$;

• •

Its Burer-Monteiro factorization with rank $p$ has at least one non-optimal second-order critical point.

The proof is in Appendix F.3.

4 Proof of Theorem 1

Let $m$ be such that

[TABLE]

From [Pataki, 1998, Theorem 2.1], Problem (SDP) has a minimizer with rank at most $p$. Consequently, Problems (SDP) and (Factorized SDP) have the same minimum, and if $V$ is a global minimizer of Problem (Factorized SDP), $X=VV^{T}$ is a minimizer of Problem (SDP). It therefore suffices to show that, for almost all cost matrices, Problem (Factorized SDP) has no second-order critical point which is not a global minimizer.

4.1 Overview of the proof

Our proof starts in a similar way as the one in [Boumal, Voroninski, and Bandeira, 2018]. Namely, we use the first and second-order properties of critical points to parametrize the set of “bad” cost matrices: we define an appropriate manifold $\mathcal{M}_{param}$ and a smooth map $\phi:\mathcal{M}_{param}\to\mathbb{S}^{n\times n}$ such that the set of bad cost matrices is included in $\phi(\mathcal{M}_{param})$.

Then the proofs differ. The authors of [Boumal, Voroninski, and Bandeira, 2018] show that, when $\frac{p(p+1)}{2}>m$, the dimension of their manifold $\mathcal{M}_{param}$ is strictly smaller than $\dim(\mathbb{S}^{n\times n})$, hence $\phi(\mathcal{M}_{param})$ has zero Lebesgue measure in $\mathbb{S}^{n\times n}$. On our side, we use additional properties of critical points to show that the set of bad cost matrices is actually included in the critical values of $\phi$, and not only in the range of $\phi$. This set has zero Lebesgue measure in $\mathbb{S}^{n\times n}$, from Sard’s theorem.

4.2 Details

We define $\mathcal{M}_{p}^{full}=\{V\in\mathcal{M}_{p},\mathrm{rank}(V)=p\}$. It is an open subset of $\mathcal{M}_{p}$, and therefore also a $(np-m)$-dimensional Riemannian manifold. We also define

[TABLE]

This set is a manifold, as stated in the following proposition, whose proof is in Appendix D.1.

Proposition 5**.**

The set $\mathcal{E}$ is a manifold, with dimension $np-m+\frac{(n-p)(n-p+1)}{2}$. Additionally, for any $(V,C_{2})\in\mathcal{E}$,

[TABLE]

We define

[TABLE]

The following lemma, whose proof is in Subsection 4.3, says that any cost matrix for which a non-optimal second-order critical point exists is a critical value of $\phi$.

Lemma 1**.**

For any cost matrix $C\in\mathbb{S}^{n\times n}$, if Problem (Factorized SDP) has a non-optimal second-order critical point, then there exist $(V,C_{2})\in\mathcal{E},\mu\in\mathbb{R}^{m}$ such that

[TABLE]

and the mapping $d\phi((V,C_{2}),\mu):T_{(V,C_{2})}\mathcal{E}\times\mathbb{R}^{m}\to\mathbb{S}^{n\times n}$ is not surjective.

From Sard’s theorem, we can therefore conclude that the set of such cost matrices has zero measure in $\mathbb{S}^{n\times n}$.

4.3 Proof of Lemma 1

Let $C\in\mathbb{S}^{n\times n}$ be a cost matrix for which a non-optimal second-order critical point exists. Let $V\in\mathcal{M}_{p}$ be such a critical point. From [Boumal, Voroninski, and Bandeira, 2018, Theorem 1.6], $\mathrm{rank}(V)=p$, so $V$ is in $\mathcal{M}_{p}^{full}$.

As $V$ is first-order critical, there exist, from Proposition 3, $\mu\in\mathbb{R}^{m},C_{2}\in\mathbb{S}^{n\times n}$ such that $C_{2}V=0$ and

[TABLE]

Let us now show that $d\phi((V,C_{2}),\mu)$ is not surjective. From [Boumal, Voroninski, and Bandeira, 2018, Theorem 1.6], the dimension of the face of $\mathcal{C}$ containing $VV^{T}$ is at least

[TABLE]

Let $X_{Face}\neq VV^{T}$ be an element of this face. Using the geometrical properties of $\{X\in\mathbb{S}^{n\times n},X\succeq 0\}$, one can establish the following proposition, whose proof is in Appendix D.2.

Proposition 6**.**

There exists $T\in\mathbb{S}^{p\times p}$ such $X_{Face}=VTV^{T}$.

Let $T$ be as in the proposition. For any $(\dot{V},\dot{C}_{2})\in T_{(V,C_{2})}\mathcal{E},\dot{\mu}\in\mathbb{R}^{m}$,

[TABLE]

Equality $(a)$ is true because $X_{face}$ belongs to $\mathcal{C}$, so $\mathcal{A}(X_{Face})=b=\mathcal{A}(VV^{T})$. Equality $(b)$ is true because of Equation (4.2), and equality $(c)$ because $C_{2}V=0_{n,p}$ from the definition of $\mathcal{E}$.

This shows that the range of $d\phi((V,C_{2}),\mu)$ in $\mathbb{S}^{n\times n}$ is included in $(X_{Face}-VV^{T})^{\perp}$, so that $d\phi((V,C_{2}),\mu)$ cannot be surjective.

5 Proof of Theorem 2

This section is devoted to the proof of the main theorem. The first two subsections, 5.1 and 5.2, each present the outline of one half of the proof, with the technical details hidden into lemmas. The remaining subsections contain the proofs of these lemmas.

5.1 First part

In the first part, we assume (proving this assumption is done in the second part) that there exists one cost matrix, $C$, for which Problem (SDP) has a unique global minimizer, with rank $r$, but Problem (Factorized SDP) has a spurious second-order critical point. We show that, for all matrices close enough to $C$, these properties still hold, hence they hold on a whole set, with non-zero Lebesgue measure.

As stated, this assertion may not be quite true ($C$ might be an isolated “bad” cost matrix). However, it becomes true if we assume $C$ to satisfy some additional non-degeneracy properties. Consequently, in this part of the proof, we admit the following lemma.

Lemma 2**.**

There exists a cost matrix $C\in\mathbb{S}^{n\times n}$ such that

• •

Problem (SDP) has a unique global minimizer, whose rank is $r$.

• •

Strict complementary slackness holds.

• •

Problem (Factorized SDP) has a second-order critical point, which is not a global minimizer.

• •

This second-order critical point is non-degenerate.

We respectively denote $X_{0}$ and $V$ the global minimizer and second-order critical point of Lemma 2. The two properties we must show are stated in the following lemmas.

Lemma 3**.**

For any matrix $C^{\prime}$ close enough to $C$, Problem (SDP) has a unique global minimizer, and this minimizer has rank $r$.

Lemma 4**.**

For any matrix $C^{\prime}$ close enough to $C$, Problem (Factorized SDP) has a second-order critical point which is not a global minimizer.

To prove Lemma 3, we use general convexity and continuity arguments to show that at least one minimizer exists, and that it goes to $X_{0}$ when $C^{\prime}$ goes to $C$. In particular, it has rank at least $r$ when $C^{\prime}$ is close enough to $C$. With another continuity argument, we show that, because strict complementary slackness holds for $C$, it also holds for any $C^{\prime}$ close enough to $C$. Therefore, the minimizer is unique (from Proposition 2), and strict complementary slackness also allows us to prove that it has rank exactly $r$. A detailed proof is in Subsection 5.3.

For Lemma 4, it actually suffices to show that, for $C^{\prime}$ close to $C$, Problem (Factorized SDP) has a second-order critical point close to $V$. Indeed, no matrix close enough to $V$ can be a global minimizer. If the Hessian at $V$ was positive definite, this would follow from general geometric arguments. But because of the invariance of the problem to multiplication by elements of $O(p)$, the Hessian is not positive definite. We must therefore consider the quotient manifold $\mathcal{M}_{p}/O(p)$ and a quotiented version of Problem (Factorized SDP). For this version, as $V$ is non-degenerate, the Hessian is positive definite, so the general arguments apply. The details are in Subsection 5.4.

5.2 Second part: proof of lemma 2

We recall that we want to construct a cost matrix $C$ such that

Problem (SDP) has a unique global minimizer, with rank $r$. 2. 2.

Strict complementary slackness holds. 3. 3.

Problem (Factorized SDP) has a second-order critical point, which is not a global minimizer. 4. 4.

This second-order critical point is non-degenerate.

It turns out that, for any rank $r$ matrix $X_{0}\in\mathcal{C}$ and any $V\in\mathcal{M}_{p}$, provided that $X_{0}$ is extremal in $\mathcal{C}$ and $\mathcal{M}_{p}$ is $X_{0}$-minimally secant at $V$, it is possible to construct a matrix $C$ as desired, and such that in addition, the unique global minimizer is precisely $X_{0}$, and the spurious critical point is precisely $V$.

Let us fix $X_{0},V$ as described (we have made the hypothesis they existed) and explain how to construct $C$. First, the results in Subsection 2.2 allow us to rephrase the desired conditions in more analytical terms: they are equivalent to the existence of $g_{1},g_{2}\in\mathbb{R}^{m},C_{1},C_{2}\in\mathbb{S}^{n\times n}$ such that

[TABLE]

The construction now proceeds as follows:

We set $g_{2}=0$. 2. 2.

We construct $g_{1},C_{1},C_{2}$ such that Properties (5.1b), (5.1c), (5.1d), (5.1e) and (5.1f) hold. 3. 3.

From $g_{1},C_{1},C_{2}$, we construct $g_{1}^{(mod)},C_{1}^{(mod)},C_{2}^{(mod)}$ which satisfy Property (5.1g) in addition to the previous ones. 4. 4.

We set $C=\mathcal{A}^{*}(g_{1}^{(mod)})+C_{1}^{(mod)}$; it satisfies all the required properties.

Points 1 and 4 are straightforward. For Points 2 and 3, see Subsections 5.5 and 5.6.

5.3 Proof of Lemma 3

To establish the lemma, it suffices to show that, for any sequence $(C^{\prime}_{k})_{k\in\mathbb{N}}$ of cost matrices converging to $C$, Problem (SDP) with cost matrix $C^{\prime}_{k}$ has a unique minimizer, and this minimizer has rank $r$, as soon as $k$ is large enough. Let $(C^{\prime}_{k})_{k\in\mathbb{N}}$ be such a sequence.

The following proposition shows that, for $k$ large enough, at least one minimizer exists, and it is arbitrarily close to $X_{0}$. Its proof is in Appendix E.1.

Proposition 7**.**

Let $\epsilon>0$ be fixed. For $k$ large enough,

• •

Problem (SDP) (with cost matrix $C^{\prime}_{k}$) admits at least one minimizer;

• •

all minimizers of Problem (SDP) belong to the ball $B(X_{0},\epsilon)$.

For any $k$ large enough, let $X^{\prime}_{k}$ be a minimizer corresponding to the cost matrix $C^{\prime}_{k}$. If there are several of them, we choose $X^{\prime}_{k}$ as an extremal point of the set of minimizers (such a point exists because the set is bounded, from Proposition 7, convex and closed); it is then also an extremal point of the feasible set $\mathcal{C}$. Let us show that, for $k$ large enough,

[TABLE]

Let $g_{1},C_{1}$ be defined as in Proposition 1: $C=\mathcal{A}^{*}(g_{1})+C_{1},C_{1}\succeq 0$ and $C_{1}X_{0}=0$. Similarly, let, for any $k$, $h_{k}\in\mathbb{R}^{m},D_{k}\in\mathbb{S}^{n\times n}$ be such that $C^{\prime}_{k}=\mathcal{A}^{*}(h_{k})+D_{k},D_{k}\succeq 0$ and $D_{k}X^{\prime}_{k}=0$.

The following lemma states that $D_{k}\overset{k\to+\infty}{\to}C_{1}$ and $h_{k}\overset{k\to+\infty}{\to}{g_{1}}$. Its proof is in Subsection E.2 and relies on the $p$-regularity of $(\mathcal{A},b)$.

Lemma 5**.**

When $k$ goes to infinity, $D_{k}\to C_{1}$ and $h_{k}\to g_{1}.$

For any $k$, because $D_{k}X^{\prime}_{k}=0$,

[TABLE]

From Proposition 7, $(X^{\prime}_{k})_{k\in\mathbb{N}}$ converges to $X_{0}$, and from Lemma 5, $(D_{k})_{k\in\mathbb{N}}$ converges to $C_{1}$. In particular, for $k$ large enough,

[TABLE]

Combined with Equation (5.3), this proves that, for $k$ large enough, $\mathrm{rank}(X^{\prime}_{k})=r\quad\mbox{and}\quad\mathrm{rank}(D_{k})=n-r.$ This establishes the first part of Property (5.2). The second part is a direct consequence of Proposition 2.

5.4 Proof of Lemma 4

We recall that $X_{0}$ is a global minimizer of Problem (SDP), but $VV^{T}$ is not: $\left\langle C,X_{0}\right\rangle<\left\langle C,VV^{T}\right\rangle$. By continuity, there is actually a neighborhood $\mathcal{V}$ of $V$ in $\mathcal{M}_{p}$ such that

[TABLE]

By continuity again, Equation (5.4) stays true if we replace $C$ by any close enough matrix $C^{\prime}$. Therefore, for $C^{\prime}$ close enough to $C$, no matrix of the form $V^{\prime}V^{\prime T}$ with $V^{\prime}\in\mathcal{V}$ can be a global minimizer of Problem (SDP), hence no matrix of $\mathcal{V}$ can be a global minimizer of Problem (Factorized SDP). From this remark, if we show that, for any $C^{\prime}$ close enough to $C$, Problem (Factorized SDP) has a second-order critical point in $\mathcal{V}$, we have proved the lemma. Let us do that.

For any cost matrix $C^{\prime}\in\mathbb{S}^{n\times n}$, we denote $f_{C^{\prime}}:W\in\mathcal{M}_{p}\to\left\langle C^{\prime},WW^{T}\right\rangle\in\mathbb{R}$ the cost function of Problem (Factorized SDP). If $\mathrm{Hess}f_{C}(V)$ was positive definite, we could apply the following general proposition (proved in Appendix E.3).

Proposition 8**.**

Let $\mathcal{M}$ be a Riemannian manifold, $E$ a finite-dimensional vector space, and $f:E\times\mathcal{M}\to\mathbb{R}$ a smooth map. Let $c\in E,v\in\mathcal{M}$ be fixed. We assume that $f(c,.)$ has a second-order critical point at $v$, and that $\mathrm{Hess}(f(c,.))(v)\succ 0$.

Then, for any neighborhood $\mathcal{V}$ of $v$ in $\mathcal{M}$, the map $f(c^{\prime},.)$ has a second-order critical point in $\mathcal{V}$ for any $c^{\prime}\in E$ close enough to $c$.

However, because $f_{C}$ is invariant to right multiplication by elements of $O(p)$, the Hessian is degenerate. Therefore, before applying the proposition, we must explicitly factorize this invariance by introducing the corresponding quotient manifold. We refer to [Absil, Mahony, and Sepulchre, 2009, Section 3.4] for basic results on quotient manifolds. Specifically, let $\mathcal{M}_{p}^{full}$ be the open subset of $\mathcal{M}_{p}$ that contains its rank $p$ elements111If $\mathcal{M}_{p}$ contains rank-deficient matrices, $\mathcal{M}_{p}/O(p)$ is not a manifold, because $\{(V,VX),V\in\mathcal{M}_{p},X\in O(p)\}$ is not a submanifold of $\mathcal{M}_{p}^{2}$. We must therefore remove rank-deficient elements from $\mathcal{M}_{p}$.. The quotient $\mathcal{M}_{p}^{full}/O(p)$ is a manifold with dimension

[TABLE]

Since $O(p)$ acts by isometries on $\mathcal{M}_{p}^{full}$, $\mathcal{M}_{p}^{full}/O(p)$ inherits from the Riemannian structure of $\mathcal{M}_{p}^{full}$. We denote $Q:\mathcal{M}_{p}^{full}\to\mathcal{M}_{p}^{full}/O(p)$ the canonical projection. It is a smooth map, with surjective differential everywhere.

For any $C^{\prime}$, since $f_{C^{\prime}}$ is invariant to the action of $O(p)$, we can define its quotient, that is the (also smooth) map $f_{C^{\prime},O(p)}:\mathcal{M}_{p}^{full}/O(p)\to\mathbb{R}$ such that

[TABLE]

The following proposition (proved in Appendix E.4) shows that there is a correspondance between the critical points of $f_{C^{\prime},O(p)}$ and $f_{C^{\prime}}$.

Proposition 9**.**

Let $\mathcal{N}_{1},\mathcal{N}_{2}$ be two Riemannian manifolds, and $f:\mathcal{N}_{2}\to\mathbb{R}$ a smooth function. Let $\phi:\mathcal{N}_{1}\to\mathcal{N}_{2}$ be a smooth map with surjective differential at any point of $\mathcal{N}_{1}$.

Then, for any $v\in\mathcal{N}_{1}$, $v$ is a second-order critical point of $f\circ\phi$ if and only if $\phi(v)$ is a second-order critical point of $f$. Additionally,

[TABLE]

This proposition, applied to $\mathcal{N}_{1}=\mathcal{M}_{p}^{full},\mathcal{N}_{2}=\mathcal{M}_{p}^{full}/O(p)$, $f=f_{C^{\prime},O(p)}$ and $\phi=Q$, shows that, because $V$ is a second-order critical point of $f_{C}=f_{C,O(p)}\circ Q$, $Q(V)$ is a second-order critical point of $f_{C,O(p)}$ and

[TABLE]

In other words, $\mathrm{Hess}f_{C,O(p)}(Q(V))$ is positive definite.

We apply Proposition 8 to $E=\mathbb{S}^{n\times n},\mathcal{M}=\mathcal{M}_{p}^{full}/O(p)$ and $f:(C^{\prime},W)\in\mathbb{S}^{n\times n}\times\mathcal{M}_{p}^{full}/O(p)\to f_{C^{\prime},O(p)}(W)$: for any neighborhood $\mathcal{V}_{O(p)}$ of $Q(V)$, $f_{C^{\prime},O(p)}$ has a second-order critical point in $\mathcal{V}_{O(p)}$ if $C^{\prime}$ is close enough to $C$.

We use this property with $\mathcal{V}_{O(p)}=Q(\mathcal{V})$. For any $C^{\prime}$ close enough to $C$, $f_{C^{\prime},O(p)}$ has a second-order critical point of the form $Q(W)$, with $W\in\mathcal{V}$. Then, from Proposition 9, $f_{C^{\prime}}$ has a second-order critical point in $\mathcal{V}$.

5.5 Construction of $C$: Point 2

We must show the existence of $g_{1},C_{1},C_{2}$ such that

[TABLE]

We simplify the problem with the following proposition, proved in Appendix E.5.

Proposition 10**.**

Without loss of generality, we can assume that

[TABLE]

With this assumption, the three conditions in Equation (5.5a) are true if and only if

[TABLE]

for some $D_{1}\in\mathbb{S}^{(n-r)\times(n-r)}$ such that $D_{1}\succ 0$. And $C_{2}V=0$ if and only if

[TABLE]

for some $F_{1},F_{2},F_{3}$. Therefore, to ensure Conditions (5.5a) and (5.5b), we must only show the existence of $g_{1},D_{1},F_{1},F_{2},F_{3}$ such that $D_{1}\succ 0$ and

[TABLE]

We observe that, if these exist, $\mathcal{A}^{*}(g_{1})$ must be of the form

[TABLE]

with $G_{3}\prec 0$ (since it is a minor of $-D_{1}$). But conversely, if there exists $g_{1}$ for which Equation (5.7) is true, we can set

[TABLE]

for some $\lambda>0$ large enough, and Equation (5.6) holds. (We observe that $D_{1}\succ 0$ for $\lambda$ large enough: all its principal minors are of the form

[TABLE]

with $-G_{3}^{(sub)}$ a principal submatrix of $-G_{3}$, whose determinant is positive because $-G_{3}\succ 0$. Therefore, all principal minors of $D_{1}$ are positive if $\lambda$ is large enough.)

To conclude, we must only prove the existence of $g_{1}$ for which Equation (5.7) is true. This is a consequence of the following lemma, whose proof is in Appendix E.6 (and relies on the minimally secant property).

Lemma 6**.**

For any $R_{1}\in\mathbb{R}^{r\times p},R_{2}\in\mathbb{S}^{p\times p}$, there exist $g_{1}\in\mathbb{R}^{m},G_{1},G_{2},G_{4},G_{5}$ such that,

[TABLE]

5.6 Construction of $C$: Point 3

In this subsection, we consider $g_{1},C_{1},C_{2}$ satisfying Properties (5.1b) to (5.1f) and construct $g_{1}^{(mod)},C_{1}^{(mod)},C_{2}^{(mod)}$ which also satisfy these properties, and, in addition, Property (5.1g):

[TABLE]

with equality if and only if $\dot{V}=VA$ for some $A\in\operatorname{Anti}(p)$.

Using Proposition 10 as in the previous subsection, we assume

[TABLE]

We set

[TABLE]

for some $t\geq 0$ large. From the following proposition (proved in Appendix E.7) these definitions satisfy Properties (5.1b) to (5.1f).

Proposition 11**.**

Whatever the value of $t\geq 0$, $g_{1}^{(mod)},C_{1}^{(mod)}$ and $C_{2}^{(mod)}$ satisfy Properties (5.1b) to (5.1f).

When $t$ is large enough, it turns out that they also satisfy Equation (5.8). This is proved in two steps, each embedded in a proposition (proofs are in Appendices E.8 and E.9): first, we observe that, to prove Equation (5.8), one can look only at matrices $\dot{V}$ in some subspace of $T_{V}\mathcal{M}_{p}$. Then, with a compactness argument, we show that, for matrices in this subspace, Equation (5.8) is true.

Proposition 12**.**

Let $\mathcal{E}_{\perp}$ be the orthogonal in $T_{V}\mathcal{M}_{p}$ of $\{VA,A\in\operatorname{Anti}(p)\}$. Equation (5.8) is true if and only if

[TABLE]

Proposition 13**.**

For $t$ large enough, Equation (5.10) is true.

Acknowledgements

We thank the reviewers for their careful reading and suggestions, which allowed us to significantly simplify our results and proofs.

Appendix A Proof of basic properties

A.1 Proof of Proposition 1

The dual of Problem (SDP) is

[TABLE]

If $C_{1},g_{1}$ are as in the statement, they are dual feasible. Because of the complementary slackness condition $C_{1}X_{0}=0$, $X_{0}$ and $(C_{1},g_{1})$ are primal-dual optimal. In particular, $X_{0}$ is a solution of Problem (SDP).

Conversely, let us assume $X_{0}$ is a solution of Problem (SDP). We temporarily admit that Slater’s condition holds (that is, $\mathcal{C}$ contains a positive definite matrix). Then strong duality holds [Wolkowicz, Saigal, and Vandenberghe, 2012, page 114] and the dual problem has at least one solution $(C_{1},g_{1})$. This pair satisfies $C=\mathcal{A}^{*}(g_{1})+C_{1}$ and $C_{1}\succeq 0$, because it is dual feasible. Strong duality means that

[TABLE]

which is equivalent to $\left\langle C_{1},X_{0}\right\rangle=0$ and in turn implies $C_{1}X_{0}=0$ because $C_{1},X_{0}\succeq 0$.

To establish Slater’s condition, we assume by contradiction that it does not hold:

[TABLE]

From a hyperplane separation theorem, there exists a non-zero $M\in\mathbb{S}^{n\times n}$, and $\mu\in\mathbb{R}$ such that

[TABLE]

Equation (A.1a) is equivalent to $M\succeq 0\mbox{ and }\mu\leq 0.$ And if we fix $V\in\mathcal{M}_{p}$, we can see that Equation (A.1b) is equivalent to

[TABLE]

In particular, $\left\langle M,VV^{T}\right\rangle\leq\mu\leq 0.$ As $M\succeq 0$, this means $MV=0$. Denoting $g\in\mathbb{R}^{m}$ a vector such that $M=\mathcal{A}^{*}(g)$, we have $\mathcal{A}^{*}(g)V=0$. Therefore, for any $\dot{V}\in\mathbb{R}^{n\times p}$,

[TABLE]

which contradicts the assumption that $(\mathcal{A},b)$ is $p$-regular.

A.2 Proof of Proposition 2

We assume that strict complementary slackness holds, but $X_{0}$ is not the unique solution of Problem (SDP), and we show that $X_{0}$ is not an extremal point of $\mathcal{C}$.

Let $X_{0}^{\prime}$ be another solution. As $(C_{1},g_{1})$ is dual optimal, $X_{0}^{\prime}$ and $C_{1}$ satisfy the complementary slackness condition:

[TABLE]

that is, $\mathrm{Range}(X_{0}^{\prime})\subset\mathrm{Ker}(C_{1})=\mathrm{Range}(X_{0})$ (the last equality is because $C_{1}X_{0}=0$ and $\mathrm{rank}(X_{0})+\mathrm{rank}(C_{1})=n$).

This inclusion and the fact that $X_{0}\succeq 0$ together imply that $X_{0}+\epsilon(X_{0}^{\prime}-X_{0})\succeq 0$ for all $\epsilon\in\mathbb{R}$ close enough to [math]. And since $X_{0},X_{0}^{\prime}$ are both feasible points of Problem (SDP),

[TABLE]

for any $\epsilon\in\mathbb{R}$. Thus, $X_{0}+\epsilon(X_{0}^{\prime}-X_{0})$ is in the feasible set $\mathcal{C}$ of Problem (SDP) for any $\epsilon$ close enough to [math], and $X_{0}$ is not extremal.

A.3 Proof of Proposition 3

From [Boumal, Voroninski, and Bandeira, 2018, Eq. 7], the gradient of the cost function of Problem (Factorized SDP) at $V$ is $2\mathrm{Proj}_{V}(CV),$ where $\mathrm{Proj}_{V}:\mathbb{R}^{n\times p}\to T_{V}\mathcal{M}_{p}$ is the orthogonal projection onto $T_{V}\mathcal{M}_{p}$. Consequently, $V$ is a first-order critical point if and only if

[TABLE]

Now, $CV=\mathcal{A}^{*}(g_{2})V$ for some $g_{2}\in\mathbb{R}^{m}$ if and only if $C=C_{2}+\mathcal{A}^{*}(g_{2})$, for some $g_{2}\in\mathbb{R}^{m},C_{2}\in\mathbb{S}^{n\times n}$ such that $C_{2}V=0$.

To show that, when it exists, the pair $(g_{2},C_{2})$ is unique, we assume that there exists another pair $(g^{\prime}_{2},C^{\prime}_{2})$ satisfying the same conditions. Then $\mathcal{A}^{*}(g_{2}-g_{2}^{\prime})V=(C_{2}^{\prime}-C_{2})V=0$. The same argument as at the end of Appendix A.1 shows that $g_{2}-g_{2}^{\prime}=0$. Therefore, $g_{2}=g_{2}^{\prime}$ and $C_{2}=C_{2}^{\prime}$.

A.4 Proof of Proposition 4

For any $\dot{V}\in T_{V}\mathcal{M}_{p}$, from [Boumal, Voroninski, and Bandeira, 2018, Eq. 10],

[TABLE]

where $S=C-\mathcal{A}^{*}(\mu)$ for some $\mu\in\mathbb{R}^{m}$ such that $2SV=\mathrm{grad}f_{C}(V)=0$.

From the uniqueness of $(C_{2},g_{2})$, we have $\mu=g_{2}$ and $S=C_{2}$.

Appendix B Discussion on Definition 4

Let for the time being $X_{0}\in\mathbb{S}^{n\times n},V\in\mathbb{R}^{n\times p}$ be fixed, such that $\mathrm{rank}(X_{0})=r$. We assume Properties 1 and 2 of Definition 4 are true:

[TABLE]

We discuss when Property 3 holds. This property is equivalent to

[TABLE]

The vector spaces $\{\dot{V},\mathrm{Range}(\dot{V})\subset\mathrm{Range}(X_{0})+\mathrm{Range}(V)\}$ and $T_{V}\mathcal{M}_{p}$ contain $\{VA,A\in\operatorname{Anti}(p)\}$, and respectively have dimensions

[TABLE]

and $np-m$.

Consequently, $\left(T_{V}\mathcal{M}_{p}\cap\{\dot{V},\mathrm{Range}(\dot{V})\subset\mathrm{Range}(X_{0})+\mathrm{Range}(V)\}\right)^{\perp}$ is a subset of $\{VA,A\in\operatorname{Anti}(p)\}^{\perp}$, with dimension at most

[TABLE]

Therefore, if $\frac{p(p+1)}{2}+pr>m$, $\left(T_{V}\mathcal{M}_{p}\cap\{\dot{V},\mathrm{Range}(\dot{V})\subset\mathrm{Range}(X_{0})+\mathrm{Range}(V)\}\right)^{\perp}$ is a strict subset of $\{VA,A\in\operatorname{Anti}(p)\}^{\perp}$, Equation (B.1) does not hold and $\mathcal{M}_{p}$ cannot be $X_{0}$-minimally secant at $V$. On the contrary, if $\frac{p(p+1)}{2}+pr\leq m$, the above upper bound on the dimension is exactly equal to the dimension if $T_{V}\mathcal{M}_{p}$ is “generic enough”, hence $\mathcal{M}_{p}$ is $X_{0}$-minimally secant at $V$.

Consequently, we expect the main assumption in Theorem 2 (the existence of $X_{0},V$ such that $\mathcal{M}_{p}$ is $X_{0}$-minimally secant at $V$, in a setting where $\frac{p(p+1)}{2}+pr\leq m$) to hold for almost all $(\mathcal{A},b)$.

We however note that, although rare, there are pairs $(\mathcal{A},b)$ for which $X_{0},V$ do not exist, hence the hypothesis cannot be trivially removed. An example is as follows.

Example 3**.**

We set $r=1,p=2$. Let $m\leq n$ be arbitrary. We consider

[TABLE]

and $b=(1,0,\dots,0)$. One can check that $(\mathcal{A},b)$ is $2$-regular.

The rank-$1$ elements of $\mathcal{C}$ are exactly the matrices $X_{0}$ of the form

[TABLE]

and $\mathcal{M}_{2}$ contains all matrices of the form $V=WX$, with $X\in O(2)$ and

[TABLE]

For any rank-$1$ $X_{0}$ in $\mathcal{C}$ and $V\in\mathcal{M}_{2}$, using the above notations, one can check that

[TABLE]

is in $T_{V}\mathcal{M}_{2}$, and $\mathrm{Range}(\dot{V})\subset\mathrm{Range}(X_{0})+\mathrm{Range}(V)$. Nevertheless, $\dot{V}\neq VA$, for all $A\in\operatorname{Anti}(2)$, so Property 3 of Definition 4 does not hold (unless $w_{1}=u$, in which case Property 2 does not hold), even if $m\geq\frac{p(p+1)}{2}+pr=5$.

Appendix C Bad critical points can exist for $m<\frac{p(p+1)}{2}+pr$

As announced in Remark 3, we provide an example where the conclusions of Theorem 2 are true, but the assumption

[TABLE]

is not.

We set $r=p=2$ and $m=n=6$, $\mathcal{A}=\mathrm{diag}$ and $b=1_{6,1}$. To show that the conclusions of Theorem 2 are true, it suffices to exhibit a matrix $C$ satisfying the conditions in Lemma 2.

We set (this precise choice was suggested by numerical experiments)

[TABLE]

We also define

[TABLE]

where $e_{1},e_{2}$ are the first two vectors of the canonical basis of $\mathbb{R}^{6\times 1}$, and $G=\begin{pmatrix}V&U_{0}&e_{1}&e_{2}\end{pmatrix}\in\mathbb{R}^{6\times 6}$ is the horizontal concatenation of $V,U_{0},e_{1},e_{2}$.

With this choice, Properties (5.1a) and (5.1e) are true. We observe that

[TABLE]

hence Property (5.1f) is also valid. A computation shows that

[TABLE]

From this expression, we see that Properties (5.1b) and (5.1d) are valid, as well as Property (5.1c) because

[TABLE]

Finally, we consider Property (5.1g). Let us define the bilinear form

[TABLE]

It contains $V\left(\begin{smallmatrix}0&1\\ 1&0\end{smallmatrix}\right)$ in its kernel (since $C_{2}V=0$). We can numerically compute the matrix associated to $q$ in an orthonormal basis of the $6$-dimensional space $T_{V}\mathcal{M}_{2}$ and check that it has $5$ strictly positive eigenvalues. Therefore, Property (5.1g) is also true.

To summarize, Properties (5.1a) to (5.1g) are all true. The matrix $C$ therefore satisfies the properties required in Lemma 2.

Appendix D Auxiliary results for the proof of Theorem 1

D.1 Proof of Proposition 5

Let $(V,C_{2})$ belong to $\mathcal{E}$. We are going to exhibit a parametrization of $\mathcal{E}$ around $(V,C_{2})$. Because $V$ has rank $p$, there exist a neighborhood $\mathcal{V}$ of $V$ in $\mathbb{R}^{n\times p}$ and a smooth map $W\in\mathcal{V}\to U_{W}\in\mathbb{R}^{n\times(n-p)}$ such that, for any $W\in\mathcal{V}$, the columns of $U_{W}$ form an orthonormal basis of $\mathrm{Range}(W)^{\perp}$. We define

[TABLE]

This function is smooth. At any point $(W,D)$, its differential with respect to $D$ is surjective: for any $(A,B)\in\mathbb{S}^{p\times p}\times\mathbb{R}^{(n-p)\times p}$, one can check that $d_{D}\psi(W,D)\cdot\dot{D}=(A,B)$ if one sets

[TABLE]

Therefore, from [Absil, Mahony, and Sepulchre, 2009, Proposition 3.3.3], $\psi^{-1}(0_{p,p},0_{n-p,p})$ is a submanifold of $\mathcal{M}_{p}^{full}\times\mathbb{S}^{n\times n}$, with dimension

[TABLE]

For any $(W,D)\in(\mathcal{M}_{p}^{full}\cap\mathcal{V})\times\mathbb{S}^{n\times n}$, the following equivalences are true:

[TABLE]

Consequently, $\psi^{-1}(0_{p,p},0_{n-p,p})$ and $\mathcal{E}$ coincide in a neighborhood of $(V,C_{2})$, which implies that $\mathcal{E}$ is also an $(np-m+\frac{(n-p)(n-p+1)}{2})$-dimensional manifold.

Its tangent space at $(V,C_{2})$ is

[TABLE]

D.2 Proof of Proposition 6

Let $U_{V}\in\mathbb{R}^{n\times p},U_{V}^{\perp}\in\mathbb{R}^{n\times(n-p)}$ be matrices whose columns respectively form an orthonormal basis of $\mathrm{Range}(V)$ and of $\mathrm{Ker}(VV^{T})=\mathrm{Range}(V)^{\perp}$. Let $G\in\mathbb{R}^{p\times p}$ be the unique matrix such that $U_{V}=VG$.

Because $\mathcal{C}\subset\{X\in\mathbb{S}^{n\times n},X\succeq 0\}$, the face of $\mathcal{C}$ containing $VV^{T}$ is a subset of the face of $\{X\in\mathbb{S}^{n\times n},X\succeq 0\}$ containing $VV^{T}$, which is, from [Laurent and Rendl, 2005, Section 2.4],

[TABLE]

Therefore, $\mathrm{Ker}(VV^{T})\subset\mathrm{Ker}(X_{Face})$, which implies

[TABLE]

Appendix E Auxiliary results for the proof of Theorem 2

E.1 Proof of Proposition 7

It suffices to show the following property:

[TABLE]

Indeed, in this case, for $k$ large enough, any minimizer of $\left\langle C^{\prime}_{k},.\right\rangle$ on the compact set $\mathcal{C}\cap\overline{B}(X_{0},\epsilon)$ (there is at least one) is a minimizer of $\left\langle C^{\prime}_{k},.\right\rangle$ on $\mathcal{C}$, and every minimizer of $\left\langle C^{\prime}_{k},.\right\rangle$ on $\mathcal{C}$ is in $B(X_{0},\epsilon)$.

We assume, by contradiction, that Property (E.1) is not true. Up to replacing $(C^{\prime}_{k})_{k\in\mathbb{N}}$ by a subsequence, we can assume that, for any $k\in\mathbb{N}$,

[TABLE]

For any $k$, let $X^{\prime}_{k}$ be such a matrix.

By compactness, we can assume that $\left((X^{\prime}_{k}-X_{0})/||X^{\prime}_{k}-X_{0}||\right)_{k\in\mathbb{N}}$ converges to some unit-normed limit $Z\in\mathbb{S}^{n\times n}$. From Equation (E.2) and because $(C^{\prime}_{k})_{k\in\mathbb{N}}$ converges to $C$, $\left\langle C,Z\right\rangle\leq 0.$ Equivalently,

[TABLE]

Observe that $X_{0}+\epsilon Z$ belongs to $\mathcal{C}$: it is the limit of the sequence

[TABLE]

Each element of this sequence belongs to $\mathcal{C}$ ($X_{0}$ and $X^{\prime}_{k}$ do, and $\mathcal{C}$ is convex), and $\mathcal{C}$ is closed, so the limit also belongs to $\mathcal{C}$. Consequently, Equation (E.3) contradicts the fact that $X_{0}$ is the unique minimizer of $\left\langle C,.\right\rangle$ on $\mathcal{C}$.

E.2 Proof of Lemma 5

As $D_{k}+\mathcal{A}^{*}(h_{k})=C^{\prime}_{k}\overset{k\to+\infty}{\to}C=C_{1}+\mathcal{A}^{*}(g_{1})$,

[TABLE]

In particular, if $h_{k}\overset{k\to+\infty}{\to}g_{1}$, then $D_{k}\overset{k\to+\infty}{\to}C_{1}$, so we only have to show that $(h_{k})_{k\in\mathbb{N}}$ converges to $g_{1}$.

By contradiction, we assume that $h_{k}\not\to g_{1}$. Up to replacing $(h_{k})_{k\in\mathbb{N}}$ by a subsequence, we can assume that $(||h_{k}-g_{1}||)_{k\in\mathbb{N}}$ is lower bounded by a positive constant and that $((h_{k}-g_{1})/||h_{k}-g_{1}||)_{k\in\mathbb{N}}$ converges to some non-zero limit $g$.

From Equation (E.4),

[TABLE]

From Proposition 7, $(X^{\prime}_{k})_{k\in\mathbb{N}}$ converges to $X_{0}$, so $C_{1}X^{\prime}_{k}\overset{k\to+\infty}{\to}C_{1}X_{0}=0,$ and because $(||h_{k}-g_{1}||)_{k\in\mathbb{N}}$ is bounded away from zero, this implies

[TABLE]

Recalling that, from the definition of $D_{k}$, $D_{k}X^{\prime}_{k}=0$ for all $k$, Equation (E.5) yields:

[TABLE]

Therefore, we also have $\mathcal{A}^{*}(g)V_{0}$, if we fix $V_{0}\in\mathbb{R}^{n\times p}$ such that $X_{0}=V_{0}V_{0}^{T}$ (it is possible, as $X_{0}\succeq 0$ and $\mathrm{rank}(X_{0})=r\leq p$). The matrix $V_{0}$ is in $\mathcal{M}_{p}$. Applying the same argument as at the end of Appendix A.1, we reach a contradiction.

E.3 Proof of Proposition 8

If we compose $f$ with a diffeomorphism along the second coordinate, we can assume that $\mathcal{M}$ is an open subset of $\mathbb{R}^{d}$, for some integer $d$ (from Proposition 9, the critical points of the composition of a function and a diffeomorphism are exactly the image by the reciprocal diffeomorphism of the critical points of the function).

Let $\mathcal{V}$ be a neighborhood of $v$ in $\mathcal{M}$. We define

[TABLE]

This is a smooth map; it satisfies $\chi(c,v)=0$ (since $v$ is a critical point of $f(c,.)$) and its differential at $(c,v)$ along the second coordinate is invertible (it is $\mathrm{Hess}(f(c,.))(v)$, which is positive definite by assumption). From the implicit function theorem, there exist a neighborhood $\mathcal{E}$ of $c$ in $E$, and a smooth function $\delta:\mathcal{E}\to\mathcal{V}$ such that $\delta(c)=v$ and $\chi(c^{\prime},\delta(c^{\prime}))=0$ for any $c^{\prime}\in\mathcal{E}$. We fix such $\mathcal{E},\delta$. Then, for any $c^{\prime}\in\mathcal{E}$,

[TABLE]

Equivalently, $\delta(c^{\prime})$ is a first-order critical point of $f(c^{\prime},.)$. Additionally, the map $c^{\prime}\to\mathrm{Hess}(f(c^{\prime},.))(\delta(c^{\prime}))$ is continuous ($f$ and $\delta$ are smooth), and

[TABLE]

As a consequence, for any $c^{\prime}\in\mathcal{E}$ close enough to $c$,

[TABLE]

Therefore, for any $c^{\prime}$ close enough to $c$, $\delta(c^{\prime})$, which is an element of $\mathcal{V}$, is a second-order critical point of $f(c^{\prime},.)$.

E.4 Proof of Proposition 9

Let $v$ belong to $\mathcal{N}_{1}$. We have

[TABLE]

As $d\phi(v)^{*}$ is injective (it is the adjoint of a surjective map), $v$ is a first-order critical point of $f\circ\phi$ if and only if $\phi(v)$ is a first-order critical point of $f$.

In this case, the Hessians of $f$ and $f\circ\phi$ at $\phi(v)$ and $v$ are linked by the following relation:

[TABLE]

As $d\phi(v)$ is surjective, $\mathrm{Hess}(f\circ\phi)(v)$ and $\mathrm{Hess}f(\phi(v))$ have the same rank, and $\mathrm{Hess}(f\circ\phi)(v)$ is positive semidefinite if and only if $\mathrm{Hess}f(\phi(v))$ is, meaning that $v$ is a second-order critical point of $f\circ\phi$ if and only if $\phi(v)$ is a second-order critical point of $f$.

E.5 Proof of Proposition 10

Let us consider for a moment an arbitrary invertible matrix $G\in\mathbb{R}^{n\times n}$, and define

[TABLE]

We denote $\tilde{\mathcal{M}}_{p}$ the set of feasible points for Problem (Factorized SDP) where $\mathcal{A}$ and $b$ have been replaced with $\tilde{\mathcal{A}}$ and $\tilde{b}$:

[TABLE]

The pair $(\tilde{\mathcal{A}},\tilde{b})$ is $p$-regular (because $(\mathcal{A},b)$ is) and one can check that $\tilde{\mathcal{M}}_{p}$ is $\tilde{X}_{0}$-minimally secant at $\tilde{V}$ (because $\mathcal{M}_{p}$ is $X_{0}$-minimally secant at $V$).

Now imagine that we can construct $\tilde{C}_{1},\tilde{g}_{1},\tilde{C}_{2},\tilde{g}_{2}$ satisfying Conditions (5.1b) to (5.1g) (with $\tilde{X}_{0},\tilde{V},\tilde{\mathcal{A}},\tilde{\mathcal{M}}_{p}$ in place of their non-tilde versions). Then, if we define

[TABLE]

we see that these objects satisfy Conditions (5.1b) to (5.1g) (with the non-tilde versions this time). Therefore, if we are able to construct $\tilde{C}_{1},\tilde{g}_{1},\tilde{C}_{2},\tilde{g}_{2}$ satisfying Conditions (5.1b) to (5.1g), it proves the existence of $C_{1},g_{1},C_{2},g_{2}$ satisfying these same conditions.

To conclude, it suffices to show that, if we properly define $G$, then

[TABLE]

Let $U_{0}\in\mathbb{R}^{n\times r}$ be such that $X_{0}=U_{0}U_{0}^{T}$ (it exists: $X_{0}$ is semidefinite positive and has rank $r$). We define

[TABLE]

where $W\in\mathbb{R}^{n\times(n-r-p)}$ is any matrix that makes $G$ invertible (it exists, as the columns of $U_{0}$ and $V$ are linearly independent, from Properties 1 and 2 of Definition 4). Equation (E.6) holds.

E.6 Proof of Lemma 6

We define $L:\mathbb{S}^{n\times n}\to\mathbb{R}^{r\times p}\times\mathbb{S}^{p\times p}$ the linear map such that, for any $R_{1},R_{2},G_{1},G_{2},G_{4},G_{5}$,

[TABLE]

Proving the lemma amounts to showing that $L\circ\mathcal{A}^{*}$ is surjective. Equivalently, it suffices to show that the dual map $\mathcal{A}\circ L^{*}$ is injective. Let $(R_{1},R_{2})$ be in its kernel:

[TABLE]

We recall that we have assumed, following Proposition 10,

[TABLE]

Therefore, if we set

[TABLE]

we have $\mathrm{Range}(\dot{V})\subset\mathrm{Range}(X_{0})+\mathrm{Range}(V)$ and $\mathcal{A}(V\dot{V}^{T}+\dot{V}V^{T})=0$, hence $\dot{V}\in T_{V}\mathcal{M}_{p}$. From Property 3 of Definition 4, there exists $A\in\operatorname{Anti}(p)$ such that $\dot{V}=VA$. As a consequence, $R_{1}=0_{r,p}$ and $R_{2}$ is both symmetric and antisymmetric, hence $R_{2}=0_{p,p}$. This proves that $\mathrm{Ker}(\mathcal{A}\circ L^{*})=\{(0_{r,p},0_{p,p})\}$, which is what we needed.

E.7 Proof of Proposition 11

Properties (5.1b), (5.1c), (5.1e) and (5.1f) are a direct consequence of Equation (5.9) and of the fact that $g_{1},C_{1},C_{2}$ satisfy these same properties.

For Property (5.1d), we have $\mathrm{rank}(C_{1}^{(mod)})\geq\mathrm{rank}(C_{1})=n-\mathrm{rank}(X_{0})$, since adding a semidefinite positive matrix to another one cannot decrease the rank. Additionally, as $C_{1}^{(mod)}X_{0}=0$ (Property (5.1c)), we also have $\mathrm{rank}(C_{1}^{(mod)})\leq n-\mathrm{rank}(X_{0})$ and, therefore, $\mathrm{rank}(C_{1}^{(mod)})=n-\mathrm{rank}(X_{0})$.

E.8 Proof of Proposition 12

Equation (5.8) naturally implies (5.10). Let us assume that Equation (5.10) is true and show the converse.

Let $\dot{V}$ be in $T_{V}\mathcal{M}_{p}$. We must show that $\left\langle C_{2}^{(mod)},\dot{V}\dot{V}^{T}\right\rangle\geq 0$, with equality if and only if $\dot{V}=VA$ for some $A\in\operatorname{Anti}(p)$. We write

[TABLE]

Using at the last line the fact that $C_{2}^{(mod)}V=0$ (Property (5.1f)), we see that

[TABLE]

Therefore, from Equation (5.10), $\left\langle C_{2}^{(mod)},\dot{V}\dot{V}^{T}\right\rangle\geq 0$, with equality if and only if $\dot{W}=0$, that is $\dot{V}\in\{VA,A\in\operatorname{Anti}(p)\}$.

E.9 Proof of Proposition 13

First, we observe that for any $\dot{V}\in\mathcal{E}_{\perp}-\{0\}$,

[TABLE]

because it is the scalar product of two semidefinite positive matrices. It is zero if and only if the last $n-r-p$ rows of $\dot{V}$ are zero, that is

[TABLE]

Because $\mathcal{M}_{p}$ is $X_{0}$-minimally secant at $V$, this is possible only if $\dot{V}=VA$ for some $A\in\operatorname{Anti}(p)$, which contradicts the fact that $\dot{V}$ is in $\mathcal{E}_{\perp}-\{0\}$. Therefore,

[TABLE]

We set $\mathcal{B}_{\perp}=\{\dot{V}\in\mathcal{E}_{\perp},||\dot{V}||_{F}=1\}$. From the previous remark and because $\mathcal{B}_{\perp}$ is compact, there exists $\epsilon>0$ such that

[TABLE]

We define $\gamma=\inf_{\dot{V}\in\mathcal{B}_{\perp}}\left\langle C_{2},\dot{V}\dot{V}^{T}\right\rangle$. For any $t$ such that $\gamma+t\epsilon>0$, it holds:

[TABLE]

In this case, by homogeneity, Equation (5.10) also holds.

Appendix F Proofs for Subsection 3.3

F.1 Proof of Corollary 1

The first part of the corollary is a direct consequence of Theorem 1, so we focus on the second one. Let $p,n$ be such that $p(p+1)/2+p\leq n$. It suffices to check the three hypotheses of Theorem 2. The first two are classical, setting

[TABLE]

Let us construct $V\in\mathcal{M}_{p}$ such that $\mathcal{M}_{p}$ is $X_{0}$-minimally secant at $V$. In Definition 4, the only delicate part is Property 3. We do not have a better method to check it that direct computation. Hence, we must choose $V$ as simple as possible, so that the equations defining $T_{V}\mathcal{M}_{p}$ and $\{\dot{V}\in\mathbb{R}^{n\times p},\mathrm{Range}(\dot{V})\subset\mathrm{Range}(U_{0})+\mathrm{Range}(V)\}$ are relatively easy to manipulate. This matrix must satisfy two constraints: it has to be in $\mathcal{M}_{p}$ (that is, all its rows must have norm $1$) and it must have at least $\frac{p(p+1)}{2}+p$ different lines (otherwise, one can check that the aforementioned equations are degenerate).

The simplest matrix $V$ that satisfies these constraints is arguably the following one: for any $i\leq p$ and $j\in\{i+1,\dots,p\}$, we respectively set the $i$-th, $(p+i)$-th and $(2p+\phi(i,j))$-th lines of $V$ as

[TABLE]

where $(e_{1},\dots,e_{p})$ is the canonical basis of $\mathbb{R}^{1\times p}$ and $\phi:\{i,j\mbox{ s.t. }1\leq i<j\leq p\}\to\left\{1,\dots,\frac{p(p-1)}{2}\right\}$ is an arbitrary bijection. For the last $n-\left(\frac{p(p+1)}{2}+p\right)$ lines, we choose any unit-normed elements of $\mathbb{R}^{1\times p}$.

This definition ensures that $V$ has rank $p$ (it contains $I_{p}$ as a submatrix). Moreover, $\left(\begin{smallmatrix}U_{0}&V\end{smallmatrix}\right)$ has rank $p+1$ (its $p+1$ first lines form an invertible matrix). Therefore, Properties 1 and 2 of Definition 4 hold.

We check Property 3. Let $\dot{V}\in T_{V}\mathcal{M}_{p}$ be such that

[TABLE]

Then there exists $(R,A)\in\mathbb{R}^{1\times p}\times\mathbb{R}^{p\times p}$ such that $\dot{V}=U_{0}R+VA$. We fix such $R,A$, and show that $R=0_{1,p}$ and $A$ is antisymmetric.

For any $i=1,\dots,p$, because $\dot{V}$ is in $T_{V}\mathcal{M}_{p}$,

[TABLE]

Consequently, $R=0_{1,p}$ and $\mathrm{diag}(A)=0$. Similarly, for any $1\leq i<j\leq p$,

[TABLE]

The matrix $A$ is therefore antisymmetric.

F.2 Proof of Corollary 2

The first part of the corollary is a direct consequence of Theorem 1.

Let us fix $S$ and $p\geq d$ such that $\frac{p(p+1)}{2}+pd\leq\frac{Sd(d+1)}{2}$, and prove the second part by checking that the hypotheses of Theorem 2 hold true. We have already said that $(\mathcal{A},b)$ is $p$-regular, which is the second hypothesis.

We set

[TABLE]

Then $X_{0}$ is an extreme point of $\mathcal{C}$ with rank $d$: the first hypothesis holds.

We show the third hypothesis by exhibiting $V$ such that $\mathcal{M}_{p}$ is $X_{0}$-minimally secant at $V$. We did not find a general construction that would be applicable for any value of $d$. Hence, we present separate constructions for the cases $d=1$, $d=2$ and $d=3$.

The case $d=1$ has already been studied in Subsection F.1, so we consider the case where $d=2$. We define the following blocks:

[TABLE]

We distinguish depending on the congruency of $p$ modulo $3$. If $p\equiv 0[3]$, for any $q=1,\dots,p/3$, we set

[TABLE]

For any $q,q^{\prime}\in\{1,\dots,p/3\}$ such that $q<q^{\prime}$, we set

[TABLE]

We define

[TABLE]

where $V_{\frac{p^{2}+5p}{6}+1},\dots,V_{S}$ are arbitrary elements of $\mathbb{R}^{2\times p}$ such that, for all $k$, $V_{k}V_{k}^{T}=I_{2}$.

Let $\dot{V}\in T_{V}\mathcal{M}_{p}$ be such that

[TABLE]

We show that $\dot{V}=VA$ for some $A\in\mathrm{Anti}(p)$. Equation (F.2) means that there exists $R\in\mathbb{R}^{2\times p},T\in\mathbb{R}^{p\times p}$ such that

[TABLE]

We call $R_{1},\dots,R_{p/3}$ the elements of $\mathbb{R}^{2\times 3}$ and $T_{1,1},\dots,T_{p/3,p/3}$ the ones of $\mathbb{R}^{3\times 3}$ such that

[TABLE]

Because $\dot{V}$ belongs to $T_{V}\mathcal{M}_{p}$, the $k$-th $2\times 2$ diagonal block of $\left(\dot{V}V^{T}+V\dot{V}^{T}\right)$ is zero for any $k=1,\dots,S$, that is, the $k$-th block of $\dot{V}V^{T}$ is antisymmetric. Using the definitions of $U_{0}$ and $V$, this property, for $k\leq\frac{4}{3}p$, can be rewritten as

[TABLE]

For any fixed $q$, this property implies that $R_{q},T_{q,q}$, which together contain $15$ coefficients, satisfy a linear system with $12$ equality constraints (there are four possible values for $s$, and each value yields $3$ constraints). It is tedious but not difficult to check that $(R_{q},T_{q,q})$ is a solution of this linear system if and only if

[TABLE]

Now, considering the $k$-th diagonal block of $\dot{V}V^{T}$ for $k=\frac{4}{3}p+1,\dots,\frac{p^{2}+5p}{6}$, we see that, for any $1\leq q<q^{\prime}\leq\frac{p}{3}$,

[TABLE]

are also antisymmetric. Taking Equation (F.3) into acccount, we can simplify this to

[TABLE]

For any fixed $q,q^{\prime}$, this is a system of $9$ linear equations over the $18$ coordinates of $T_{q,q^{\prime}}$ and $T_{q^{\prime},q}$, which can be seen to be equivalent to

[TABLE]

Combining Equations (F.3) and (F.4) proves that $R$ is zero and $T$ is antisymmetric, thus $\dot{V}$ is indeed of the form $VA$ for some $A\in\mathrm{Anti}(p)$, which concludes the proof in the case where $p\equiv 0[3]$.

In the case where $p\equiv 1[3]$, we define $W_{q}^{(1)},W_{q}^{(2)},W_{q}^{(3)},W_{q}^{(4)}$ and $X_{q,q^{\prime}}^{(1)},X_{q,q^{\prime}}^{(2)},X_{q,q^{\prime}}^{(3)}$ as previously, for $q\leq\frac{p-4}{3}$, and $q<q^{\prime}\leq\frac{p-4}{3}$. For $q=\frac{p-1}{3}$, we define six matrices $(W_{q}^{(i)})_{i=1,\dots,6}$ by

[TABLE]

with

[TABLE]

And for $q\leq\frac{p-4}{3},q^{\prime}=\frac{p-1}{3}$, we define the following four matrices:

[TABLE]

We define $V$ as before. Establishing that $\mathcal{M}_{p}$ is $X_{0}$-minimally secant at $V$ can be done as previously, the only difference being that we have to write $R,T$ as

[TABLE]

with $R_{1},\dots,R_{(p-4)/3}\in\mathbb{R}^{2\times 3},R_{(p-1)/3}\in\mathbb{R}^{2\times 4}$, and

T=$$\begin{pmatrix}T_{1,1}&\dots&T_{1,\frac{p-1}{3}}\\ \vdots&&\vdots\\ T_{\frac{p-1}{3},1}&\dots&T_{\frac{p-1}{3},\frac{p-1}{3}}\end{pmatrix},3$$4$$3$$4

Finally, in the case where $p\equiv 2[3]$, we define $W_{q}^{(1)},W_{q}^{(2)},W_{q}^{(3)},W_{q}^{(4)}$ and $X^{(1)}_{q,q^{\prime}},X^{(2)}_{q,q^{\prime}},X^{(3)}_{q,q^{\prime}}$ as before for $q\leq\frac{p-2}{3}$ and $q<q^{\prime}\leq\frac{p-2}{3}$. For $q=\frac{p+1}{3}$, we define only three matrices $(W_{q}^{(i)})_{i=1,2,3}$:

[TABLE]

with

[TABLE]

For $q\leq\frac{p-2}{3}$, $q^{\prime}=\frac{p+1}{3}$, we set

[TABLE]

We conclude as before.

The case $d=3$ can be dealt with in the same way as $d=2$, but is even more technical. The easiest thing to do (although maybe not the most elegant one) is to distinguish $12$ cases, depending on the congruency of $p$ modulo $12$. To avoid pages of definitions, we only focus on the case where $p\equiv 0[12]$ and, even in this case, only provide a sketch of proof.

For any $q\leq\frac{p}{12}$, we define $19$ matrices $(W_{q}^{(i)})_{i\leq 19}$ of size $3\times p$, by

[TABLE]

for matrices $G_{1},\dots,G_{19}\in\mathbb{R}^{3\times 12}$ suitably chosen222By analycity arguments, one can see that a “generic” choice of $G_{1},\dots,G_{19}$ such that $G_{i}G_{i}^{T}=I_{3}$ works.. Then, for any $q,q^{\prime}\leq\frac{p}{12}$, with $q<q^{\prime}$, one defines $24$ matrices $(X_{q,q^{\prime}}^{(i)})_{i\leq 24}$

[TABLE]

for appropriate $G^{\prime}_{i},G^{\prime\prime}_{i}$.

If we divide $R$ and $T$ into, respectively, $3\times 12$ and $12\times 12$ blocks, we can check in the exact same way as in the case $d=2$ (using a computer to solve the linear systems) that, if $\dot{V}=U_{0}R+VT$ is in $T_{V}\mathcal{M}_{p}$, then $\dot{V}=VA$ for some $A=\mathrm{Anti}(p)$.

F.3 Proof of Corollary 3

As in the previous two subsections, it suffices to show fix $p$ such that $\frac{p(p+1)}{2}+p\leq S$ and show that the assumptions in Theorem 2 are verified. The second assumption is elementary. We show the first and third ones by deducing them from the same assumptions in the MaxCut case (Subsection F.1).

Let us set, for any $q\in\mathbb{N}$,

[TABLE]

We set

[TABLE]

It is an extreme point of $\mathcal{C}$ with rank $1$, so the first hypothesis holds true.

To establish the third hypothesis, let $V^{(MaxCut)}\in\mathbb{R}^{S\times p}$ be the matrix defined in Subsection F.1 (with $n=S$). We set

[TABLE]

If $\dot{V}$ is an element of $T_{V}\mathcal{M}_{p}$ such that

[TABLE]

we check that $\dot{V}$ must be of the form $\mathrm{Ins}_{p}(\dot{V}^{(MaxCut)})$ for some $\dot{V}^{(MaxCut)}\in T_{V}\mathcal{M}^{(MaxCut)}_{p}$ such that

[TABLE]

(Here, $\mathcal{M}_{p}^{(MaxCut)}$ is the feasible set of Problem (Factorized SDP) in the MaxCut case, for $n=S$, and we define $U_{0}^{(MaxCut)}=1_{S,1}$.)

From the proof in Subsection F.1, there exists an antisymmetric $A\in\mathrm{Anti}(p)$ such that $\dot{V}^{(MaxCut)}=V^{(MaxCut)}A$. For this $A$, we have

[TABLE]

Therefore, $\mathcal{M}_{p}$ is $X_{0}$-minimally secant at $V$, which concludes the proof.