Uniqueness of nonnegative matrix factorizations by rigidity theory

TL;DR

This paper applies rigidity theory to analyze the uniqueness of nonnegative matrix factorizations, establishing conditions for local and global rigidity and extending results to completely positive factorizations.

Contribution

It introduces a novel approach using rigidity theory to characterize the uniqueness of nonnegative matrix factorizations when nonnegative rank equals rank.

Findings

Characterizes infinitesimally rigid nonnegative factorizations.

Shows local rigidity implies global rigidity under certain conditions.

Provides the strongest known necessary condition for factorization uniqueness.

Abstract

Nonnegative matrix factorizations are often encountered in data mining applications where they are used to explain datasets by a small number of parts. For many of these applications it is desirable that there exists a unique nonnegative matrix factorization up to trivial modifications given by scalings and permutations. This means that model parameters are uniquely identifiable from the data. Rigidity theory of bar and joint frameworks is a field that studies uniqueness of point configurations given some of the pairwise distances. The goal of this paper is to use ideas from rigidity theory to study uniqueness of nonnegative matrix factorizations in the case when nonnegative rank of a matrix is equal to its rank. We characterize infinitesimally rigid nonnegative factorizations, prove that a nonnegative factorization is infinitesimally rigid if and only if it is locally rigid and a certain matrix achieves its maximal possible Kruskal rank, and show that locally rigid nonnegative factorizations can be extended to globally rigid nonnegative factorizations. These results give so far the strongest necessary condition for the uniqueness of a nonnegative factorization. We also explore connections between rigidity of nonnegative factorizations and boundaries of the set of matrices of fixed nonnegative rank. Finally we extend these results from nonnegative factorizations to completely positive factorizations.

Tables

Table 1. Table 1 : Number of zero patterns with 13 13 13 zeros satisfying the conditions of Theorem  3.4 and Lemma  3.9 for different matrix sizes such that every row of A 𝐴 A contains a zero, and the number of columns of B 𝐵 B is five or every column of B 𝐵 B contains a zero.

$5\times 5$	$6\times 5$	$6\times 6$	$7\times 5$	$7\times 6$	$8\times 5$	$9\times 5$
15	26	14	24	11	10	2

Equations

$$\frac{d}{dt}\Vert p_i-p_j{\Vert }^2=0\text{ for all }(i,j)\in E.$$

$$(p_i-p_j{)}^T({\dot{p}}_i-{\dot{p}}_j)=0\text{ for all }(i,j)\in E,$$

$$a_i^T{\dot{b}}_j+{\dot{a}}_i^T{b}_j=0\text{ for all }(i,j)\in E$$

$$\mu :{\mathbb{R}}^{m\times r}\times {\mathbb{R}}^{r\times n}\to {\mathbb{R}}^{m\times n}$$

$$a_i^T{\dot{b}}_j+{\dot{a}}_i^T{b}_j=0\text{ for }(i,j)\in [m]\times [n],$$

$$a_i+t{\dot{a}}_i\ge 0\text{ for }i\in [m]\text{ and }t\in [0,ϵ),b_j+t{\dot{b}}_j\ge 0\text{ for }j\in [n]\text{ and }t\in [0,ϵ)$$

$$W_{(A,B)}:=\{D\in {\mathbb{R}}^{r\times r}\mid \exists ϵ>0\text{ such that }A+tAD\ge 0,B-tDB\ge 0\text{ for }t\in [0,ϵ)\}.$$

$$\left(\begin{array}{ccc}0& \cdot & \cdot \\ \cdot & 0& \cdot \\ \cdot & \cdot & 0\\ \cdot & \cdot & 0\\ \cdot & \cdots & \cdot \\ ⋮& \ddots & ⋮\\ \cdot & \cdots & \cdot \end{array}\right)\left(\begin{array}{cccccc}0& \cdot & \cdot & \cdot & \cdots & \cdot \\ \cdot & 0& \cdot & ⋮& \ddots & ⋮\\ \cdot & \cdot & 0& \cdot & \cdots & \cdot \end{array}\right).$$

$$W_{(A,B)}^{\vee }=\mathrm{cone}({\mathcal{A}}_1\cup \cdots \cup {\mathcal{A}}_m\cup {\mathcal{B}}_1\cup \cdots \cup {\mathcal{B}}_n).$$

$$k|\alpha |+\ell |\beta |\le (r-|\alpha |)|\alpha |+(r-|\beta |)|\beta |-|\alpha \setminus \beta ||\beta \setminus \alpha |.$$

$$A_1=(0,1/3+\epsilon ,1/3-\epsilon ,1/3),B_1=(0,1/3,1/3+\epsilon ,1/3-\epsilon ),C_1=(0,1/3-\epsilon ,1/3,1/3+\epsilon ),$$

$$A_2=(1/3,0,1/3+\epsilon ,1/3-\epsilon ),B_2=(1/3-\epsilon ,0,1/3,1/3+\epsilon ),C_2=(1/3+\epsilon ,0,1/3-\epsilon ,1/3),$$

$$A_3=(1/3-\epsilon ,1/3,0,1/3+\epsilon ),B_3=(1/3+\epsilon ,1/3-\epsilon ,0,1/3),C_3=(1/3,1/3+\epsilon ,0,1/3-\epsilon ).$$

$$\frac{1}{(1+(0.416827-1)t)}(1/3,1/3-2t,1/3+t,0.416827t)$$

$$V_1=(0,1/3,1/3,1/3),V_2=(1/3,0,1/3,1/3),V_3=(1/3,1/3,0,1/3),\Omega =(1/3,1/3,1/3,0).$$

$$W_1=(1-3\epsilon )V_1+\epsilon V_2+\epsilon V_3+\epsilon \Omega ,W_2=\epsilon V_1+(1-3\epsilon )V_2+\epsilon V_3+\epsilon \Omega ,$$

$$W_3=\epsilon V_1+\epsilon V_2+(1-3\epsilon )V_3+\epsilon \Omega ,W=\epsilon V_1+\epsilon V_2+\epsilon V_3+(1-3\epsilon )\Omega ,$$

$$F_{11}=0.81V_2+0.01V_3+0.18\Omega ,F_{12}=0.14V_2+0.20V_3+0.66\Omega ,$$

$$F_{21}=0.43V_1+0.22V_3+0.35\Omega ,F_{22}=0.20V_1+0.49V_3+0.31\Omega ,$$

$$F_{31}=0.11V_1+0.87V_2+0.02\Omega ,F_{32}=0.43V_1+0.12V_2+0.45\Omega ,$$

$$H=1/3V_1+1/3V_2+1/3V_3.$$

$$\left(\begin{array}{cccc}0& \cdot & \cdot & \cdot \\ \cdot & 0& \cdot & \cdot \\ \cdot & \cdot & 0& \cdot \\ \cdot & \cdot & \cdot & 0\\ \cdot & \cdot & \cdot & 0\end{array}\right)\left(\begin{array}{ccccccc}0& 0& \cdot & \cdot & \cdot & \cdot & \cdot \\ \cdot & \cdot & 0& 0& \cdot & \cdot & \cdot \\ \cdot & \cdot & \cdot & \cdot & 0& 0& \cdot \\ \cdot & \cdot & \cdot & \cdot & \cdot & \cdot & 0\end{array}\right)$$

$$\frac{1}{(1+(0.416827-1)t)}(1/3,1/3-2t,1/3+t,0.416827t),$$

$$\mu :{\mathbb{R}}_r^{m\times r}\times {\mathbb{R}}_r^{r\times n}\to {\mathbb{R}}_r^{m\times n}.$$

$${\mu }^{-1}(M)=\{(AC,C^{-1}B)\mid C\in {\mathbb{R}}^{r\times r}\text{ invertible}\}.$$

$$F:=\{(C,C^{-1})\mid C\in {\mathbb{R}}^{r\times r}\text{ invertible}\}\subseteq {\mathbb{R}}^{r\times r}\times {\mathbb{R}}^{r\times r}.$$

$${\nu }_{(A,B)}:{\mathbb{R}}^{r\times r}\times {\mathbb{R}}^{r\times r}\to {\mathbb{R}}^{m\times r}\times {\mathbb{R}}^{r\times n}$$

$$(C,D)\mapsto (AC,DB)$$

$$((\alpha ,\beta ),(\gamma ,{\gamma }^{-1}))\mapsto (\alpha \gamma ,{\gamma }^{-1}\beta )$$

$$T_{(I,I)}F=\{(D,-D)\mid D\in {\mathbb{R}}^{r\times r}\}.$$

$$M=\left(\begin{array}{ccc}2& 1& 1\\ 1& 2& 1\\ 1& 1& 2\end{array}\right)=\left(\begin{array}{ccc}0& 1& 1\\ 1& 0& 1\\ 1& 1& 0\end{array}\right)\left(\begin{array}{ccc}0& 1& 1\\ 1& 0& 1\\ 1& 1& 0\end{array}\right).$$

$$\left(\begin{array}{ccc}0& 0& 0\\ 1& 0& 0\\ 1& 0& 0\end{array}\right),\left(\begin{array}{ccc}0& 1& 0\\ 0& 0& 0\\ 0& 1& 0\end{array}\right),\left(\begin{array}{ccc}0& 0& 1\\ 0& 0& 1\\ 0& 0& 0\end{array}\right),\left(\begin{array}{ccc}0& -1& -1\\ 0& 0& 0\\ 0& 0& 0\end{array}\right),\left(\begin{array}{ccc}0& 0& 0\\ -1& 0& -1\\ 0& 0& 0\end{array}\right),\left(\begin{array}{ccc}0& 0& 0\\ 0& 0& 0\\ -1& -1& 0\end{array}\right)$$

$$W_{(A,B)}=\left\{\begin{pmatrix}d_{1}&-t&t\\ t&d_{2}&-t\\ -t&t&d_{3}\end{pmatrix}\;\Bigg{|}\;t,d_{1},d_{2},d_{3}\in\mathbb{R}\right\}.$$

$$(I+tD,I+tE).$$

Full text

1 Introduction

Nonnegative matrix factorization of size $r$ decomposes a matrix $M\in\mathbb{R}^{m\times n}_{\geq 0}$ as $M=AB$ where $A\in\mathbb{R}^{m\times r}_{\geq 0}$ and $B\in\mathbb{R}^{r\times n}_{\geq 0}$. The smallest $r\in\mathbb{N}$ such that $M$ has a size-$r$ nonnegative factorization is called the nonnegative rank of $M$. Approximations by matrices of low nonnegative rank are ubiquitous in data mining applications where they are used to explain a dataset by a small number of parts; the number of parts being equal to nonnegative rank of the approximation. For example, Lee and Seung [20] used nonnegative matrix factorizations for studying databases of face images. In this application, rows of $M$ correspond to different pixels of an image and columns of $M$ correspond to different images. A size-$r$ nonnegative factorization finds $r$ basis images (corresponding to columns of $A$) such that every original image is a nonnegative linear combination of these basis images (nonnegative coefficients are given by columns of $B$). Another popular application is topic modeling [32], where the matrix $M$ gives frequencies of words in documents, and a nonnegative matrix factorization decomposes this matrix with respect to topics. Nonnegative matrix factorizations and nonnegative rank appear also in complexity theory [33], computational biology [8], music analysis [27], blind source separation [6], spectral data analysis [24]. For many of these applications it is desirable that there exists an essentially unique nonnegative factorization that explains the data, i.e. model parameters are identifiable from the data. We say essentially unique because nonnegative matrix factorizations are never completely unique: Given a nonnegative factorization $AB$ one obtains a new factorization $(AC)(C^{-1}B)$ by multiplying $A$ and $B$ by a scaling or permutation matrix and its inverse correspondingly.

The uniqueness of nonnegative factorizations was first addressed by Donoho and Stodden [10] for black and white images with $P$ parts such that each part can appear in $A$ articulations. They showed that separability and complete factorial sampling guarantee uniqueness of nonnegative matrix factorization. Separability requires that one of the factors contains the $r\times r$ identity matrix as a submatrix and complete factorial sampling requires that the database contains all $A^{P}$ images where each of the $P$ parts appears in each of the $A$ articulations. Another sufficient condition appears in the work of Gillis [15] and it requires $M$ to have $r$ nonzero columns each with $r-1$ zero entries with different sparsity patterns. Theis et al [28] prove uniqueness under a sparsity assumption on the nonnegative factors. Ding et al show that nonnegative matrix factorizations are unique assuming that one of the factors is orthogonal [9]. Many authors have established guarantees for identifiability under volume minimization or maximization of the polytope associated to one of the factors [4, 31, 13, 21, 11]. The first necessary condition was given by Laurberg et al [19] and it requires the rows of $A$ and columns of $B$ to be boundary closed. More precisely, for every $i\neq j\in[r]$ there must exist a row $a_{k}$ of $A$ such that $a_{ki}=0$ and $a_{kj}\neq 0$ (and similarly for columns of $B$). A comprehensive review on uniqueness of nonnegative matrix factorizations is given by Fu et al [12]. Despite the recent progress on uniqueness of nonnegative matrix factorizations, the current sufficient conditions are either relatively restrictive or require additional assumptions on nonnegative factors, and a little is known about necessary conditions.

The goal of this paper is to study the uniqueness of nonnegative matrix factorizations by building on the rigidity theory of bar and joint frameworks, which studies uniqueness of point configurations given some pairwise distances between the points. This approach has been already successfully adapted to investigating the uniqueness of low-rank matrix completion [26]. Similarly to rigidity theory, we define infinitesimally, locally and globally rigid nonnegative matrix factorizations. We consider the case when nonnegative rank is equal to rank. Before going into more details, we give a brief overview of the implications between these notions. Global rigidity is the same as the uniqueness of a nonnegative matrix factorization. Local rigidity is a necessary condition for global rigidity and infinitesimal rigidity is a sufficient condition for local rigidity. We give a characterization of infinitesimal rigidity that can be checked computationally (Proposition 3.3). We show that infinitesimal rigidity implies local rigidity and that a locally rigid nonnegative matrix factorization that is not infinitesimally rigid implies that the Kruskal rank of a specified matrix is not maximal (Proposition 4.8). These results lead to Algorithm 1 for determining local rigidity of a nonnegative matrix factorization (one possible output of the algorithm is that local rigidity of the matrix cannot be determined) and to a necessary condition for uniqueness of nonnegative factorizations that strengthens the necessary condition in [19, Theorem 3] (Corollary 4.10). A next step will be to use rigidity theory to study sufficient conditions for global rigidity. To do this, we believe that one has to come up with an analogue of a stress matrix in rigidity theory, similarly as we have developed an analogue of a rigidity matrix in this work.

In more detail, in the rigidity theory of frameworks and low-rank matrix completion, infinitesimal motions are required to have derivatives of pairwise distances or inner products equal to zero. A framework is called infinitesimally rigid if all its infinitesimal motions are trivial ones. Checking infinitesimal rigidity is equivalent to checking rank of a rigidity or a completion matrix. In the nonnegative matrix factorization case, infinitesimal motions are additionally required to preserve nonnegativity of factors. Now checking infinitesimal rigidity amounts to checking whether positive span of a matrix is isomorphic to a specified linear subspace of $\mathbb{R}^{r^{2}}$ (Proposition 3.3). The difference with the frameworks and low-rank matrix completion case is that instead of linear span one has to consider positive span of a rigidity matrix. Hence a linear algebra problem becomes a polyhedral geometry problem. We also give purely combinatorial necessary conditions for infinitesimal rigidity that follow from this characterization (Theorem 3.4, Lemmas 3.9 and 3.10). Infinitesimal rigidity always implies local rigidity, and although the converse is not always true as we will see in Example 4.11, then if a nonnegative factorization is locally rigid and a certain matrix achieves its maximal possible Kruskal rank, then it is infinitesimally rigid (Proposition 4.8). We also show that every locally rigid nonnegative factorization can be extended to globally rigid nonnegative factorization by adding at most $r$ strictly positive rows to $A$ and at most $r$ strictly positive columns to $B$ (Corollary 4.7).

Matrices of size $m\times n$ and nonnegative rank at most $r$ form a semialgebraic set, which we denote by $\mathcal{M}^{m\times n}_{\leq r}$. We explore connections between rigidity of nonnegative matrix factorizations and boundaries of the set $\mathcal{M}^{m\times n}_{\leq r}$. The first motivation for this is that a matrix with a unique nonnegative matrix factorization always lies on the boundary of $\mathcal{M}^{m\times n}_{\leq r}$. The second motivation is that understanding boundaries of a semialgebraic set is often easier than deriving a semialgebraic description of the set, and sometimes boundaries provide the first step towards obtaining a semialgebraic description. This was the case for matrices of nonnegative rank at most three [18]. This semialgebraic description gives an algorithm, polynomial in $m$ and $n$, to decide if a rank-three matrix has nonnegative rank three by checking one condition for each possible boundary component. A semialgebraic description of the set $\mathcal{M}^{m\times n}_{\leq r}$ would in general allow one to check directly whether a matrix has nonnegative rank at most $r$ without constructing a nonnegative factorization of the matrix. Neither boundaries nor a semialgebraic description of $\mathcal{M}^{m\times n}_{\leq r}$ is known for $r\geq 4$. Vavasis showed that computing nonnegative rank is NP-hard [30] and the best known algorithm for deciding whether an $m\times n$ matrix has nonnegative rank at most $r$ runs in time $(mn)^{O(r^{2})}$ by the work of Moitra [22]. A necessary and sufficient condition for a matrix to lie on the boundary of $\mathcal{M}^{m\times n}_{\leq 3}$ is that it contains a zero or all its size three nonnegative factorizations are infinitesimally rigid [23]. This is not true for $r>3$. Example 4.11 provides a nonnegative matrix factorization that is locally and globally rigid, and hence on the boundary, but not infinitesimally rigid. Furthermore, in Section 5.2 we will see matrices on the boundary of $\mathcal{M}^{m\times n}_{\leq r}$ with nonnegative factorizations that are not even locally rigid.

We finish the paper with extending our results to completely positive factorizations. Let $M$ be a nonnegative real symmetric matrix. The completely positive rank of $M$ is the smallest $r$ such that $M=AA^{T}$ for some nonnegative $n\times r$ matrix $A$ [1]. We consider real symmetric matrices whose completely positive rank is equal to their rank. We define infinitesimally, locally and globally rigid completely positive factorizations, and show that results analogous to the nonnegative factorizations case hold.

The outline of our paper is the following. In Section 2, we give preliminaries on rigidity theory (Section 2.1), geometric characterizations of nonnegative rank via nested polytopes (Section 2.2) and nonnegative rank boundaries (Section 2.3). In Section 3, we study infinitesimally rigid factorizations. In Section 4, we study locally rigid nonnegative factorizations. In Section 5, we study connections between rigidity and boundaries of $\mathcal{M}^{m\times n}_{\leq r}$. In Section 6, we adapt these results on nonnegative rank of general matrices, to the case of completely positive rank on symmetric matrices. In Appendix A, we show that in the case of $5\times 5$ matrices of nonnegative rank four, for every zero pattern that satisfies the necessary condition in Theorem 3.4, there exists an infinitesimally rigid nonnegative factorization $(A,B)$ that realizes the zero pattern. Code for computations in this paper is available at

https://github.com/kaiekubjas/nonnegative-rank-four-boundaries

2 Preliminaries

2.1 Rigidity theory

The goal of rigidity theory is to determine whether $n$ points in $\mathbb{R}^{d}$ can be determined uniquely up to rigid transformations (translations, rotations, reflections) given a partial set of pairwise distances between them. We will introduce rigidity theory following [26, Section 2] and discuss the connection between the rigidity theory and uniqueness of low-rank matrix completions established by Singer and Cucuringu [26, Sections 3 and 4]. This subsection can be skipped at the first reading and used as a reference.

A bar and joint framework $G(p)$ in $\mathbb{R}^{d}$ consists of a graph $G=(V,E)$, a set of distances $\{d_{ij}\in\mathbb{R}_{\geq 0}:(i,j)\in E\}$ and a set of points $p_{1},\ldots,p_{|V|}\in\mathbb{R}^{d}$ such that $\|p_{i}-p_{j}\|=d_{ij}$ for all $(i,j)\in E$. One can think of the distance constraints as bars that are joining corresponding points. Consider a motion of the bar and joint framework parametrized by $t$, i.e. $p_{i}(t)$ is the position of the $i$-th point at time $t$. To preserve the distances given by $E$, the motion has to satisfy

[TABLE]

Denoting the velocity of $p_{i}$ by $\dot{p}_{i}$ for $i=1,\ldots,|V|$, these constraints can be rewritten as

[TABLE]

or in the matrix form as $R_{G}(p)\dot{p}=0$ where $R_{G}(p)$ is a $|E|\times n|V|$ matrix and $\dot{p}=(\dot{p}^{T}_{1},\ldots,\dot{p}^{T}_{n})^{T}$. The matrix $R_{G}(p)$ is called the rigidity matrix of the bar and joint framework.

A motion satisfying Equation (2.1) is called an infinitesimal motion. Trivial motions are motions given by rotation and translation of the entire framework, also referred to as rigid transformations. A trivial motion satisfies $\dot{p}_{i}=Dp_{i}+b$ with $D\in\mathbb{R}^{d\times d}$ skew-symmetric and $b\in\mathbb{R}^{d}$, and every trivial motion is infinitesimal. A bar and joint framework is called infinitesimally rigid if all its infinitesimal motions are trivial. There are $\frac{(d-1)d}{2}$ degrees of freedom choosing a skew-symmetric matrix $D$ (rotations) and $d$ degrees of freedom choosing a vector $b$ (translations). Every trivial motion is in the kernel of the rigidity matrix $R_{G}(p)$, so the framework is infinitesimally rigid if and only if the dimension of the kernel of the rigidity matrix $R_{G}(p)$ is equal to $\frac{d(d+1)}{2}$.

A framework $G(p)$ is locally rigid if there exists a neighborhood $\mathcal{N}$ of the framework $G(p)$ such that $G(p)$ is the only framework up to rigid transformations with the same distance constraints in the neighborhood $\mathcal{N}$. A framework $G(p)$ is regular if $\operatorname{rank}R_{G}(p)=\max\{\operatorname{rank}R_{G}(q):\|q_{i}-q_{j}\|=d_{ij}\text{ for all }(i,j)\in E\}$.

Theorem 2.1** ([2]).**

A framework is infinitesimally rigid if and only if it is regular and locally rigid.

A framework is called generic if the coordinates of the points $p_{1},\ldots,p_{|V|}$ are algebraically independent over $\mathbb{Q}$. Any generic framework is regular. Theorem 2.1 implies that local rigidity is a generic property in the sense that if generic $G(p)$ is locally rigid then most frameworks $G(q)$ are locally rigid. Hence one can talk about local rigidity of graphs. This also allows one to check with probability one whether a framework is locally rigid by choosing a random configuration $p_{1},\ldots,p_{|V|}$ and checking whether the dimension of the kernel of the rigidity matrix $R_{G}(p)$ is equal to $\frac{d(d+1)}{2}$.

Finally, a framework $G(p)$ is globally rigid if all other frameworks in $\mathbb{R}^{d}$ that have the same distance constraints are related to $G(p)$ by rigid transformations. Global rigidity is also a generic property, and there are necessary and sufficient results using ranks of stress matrices for checking generic global rigidity. However, since we focus on infinitesimal and local rigidity of nonnegative factorizations in this paper, we do not present them here.

Singer and Cucuringu established a connection between the rigidity theory and low-rank matrix completion [26]. Let $M$ be a $m\times n$ matrix of rank $r$ and let $(A,B)$ give a rank-$r$ factorization of $M$. Let the rows of $A$ be $a^{T}_{1},\ldots,a^{T}_{m}\in\mathbb{R}^{r}$ and the columns of $B$ be $b_{1},\ldots,b_{n}\in\mathbb{R}^{r}$. Then $M_{ij}=a^{T}_{i}b_{j}$.

The observed entries of $M$ define a bipartite graph $G=(V,E)$ on $m+n$ vertices. The vertices $V$ correspond to $a_{1},\ldots,a_{m},b_{1},\ldots,b_{n}$ and the edges $E$ correspond to observed entries of $M$. Instead of distance constraints, one fixes inner products $M_{ij}=a^{T}_{i}b_{j}$ for $(i,j)\in E$. The graph $G$, the inner products $\{M_{ij}\in\mathbb{R}:(i,j)\in E\}$ and the points $a_{1},\ldots,a_{m},b_{1},\ldots,b_{n}\in\mathbb{R}^{r}$ define a framework. Consider a deformation of a framework parametrized by $t$. To preserve the inner products $M_{ij}=a^{T}_{i}b_{j}$ for $(i,j)\in E$, the deformation has to satisfy

[TABLE]

where $\dot{a}$ and $\dot{b}$ are velocities of $a$ and $b$. The same constraints can be written in a matrix form using the $r\times(m+n)$ completion matrix $C_{G}(a,b)$.

A deformation satisfying Equation (2.2) is called an infinitesimal deformation. A trivial deformation is one given by $\dot{a}_{i}=D^{T}a_{i}$ and $\dot{b}_{j}=-Db_{j}$ with $D\in\mathbb{R}^{r\times r}$, and every trivial deformation is infinitesimal. The framework $G(a,b)$ is called infinitesimally completable if all its infinitesimal motions are trivial. Since there are $r^{2}$ degrees of freedom choosing an invertible matrix $D$ and every trivial deformation is in the kernel of the completion matrix $C_{G}(a,b)$, then a non-trivial infinitesimal deformation exists if and only if the dimension of the kernel of the completion matrix $C_{G}(a,b)$ is equal to $r^{2}$.

A framework $G(p)$ is locally completable if there exists a neighborhood $\mathcal{N}$ of the framework $G(p)$ such that $G(p)$ is the only framework in the neighborhood $\mathcal{N}$ up to trivial deformations with the same inner products. As in the rigidity theory of bar and joint frameworks, local completability of a generic framework is equivalent to infinitesimal completability, and hence local completability is a generic property. Therefore one can talk about local completability of a bipartite graph. For the low-rank matrix completion problem this implies that although one does not know the factor matrices $A$ and $B$, one can check with probability one whether a partial matrix is locally completable by checking whether the partial matrix with the same underlying graph constructed from generic $A$ and $B$ is locally completable.

A framework is globally completable if it is the only framework up to trivial deformations giving the same inner products. Singer and Cucuringu also conjecture a sufficient condition for global completability using rank of stress matrices.

In Sections 3 and 4, we will establish the connection between rigidity theory and nonnegative matrix factorizations. Although a framework is defined similarly to the low-rank matrix completion setting, the definition of infinitesimal rigidity is different because of the nonnegativity requirement of the factorization. Essentially, a linear algebra problem becomes a convex geometry problem: Instead of computing the span of a completion matrix one has to compute the conic hull of a factorization matrix.

2.2 Geometric characterization of nonnegative rank

Nonnegative rank can be characterized geometrically via nested polyhedral cones. We describe two equivalent constructions from the literature for matrices of equal rank and nonnegative rank.

The first description is due to Cohen and Rothblum [7]. It defines $P$ as the convex cone spanned by the columns of $M$ and $Q$ as the intersection of $\mathbb{R}^{m}_{\geq 0}$ and the column span of $M$. Let $(A,B)$ be a rank-$r$ factorization of $M$, and let $\Delta$ be the simplicial cone spanned by the columns of $A$. Since $A$ and $M$ have the same column span, the cones $P$, $\Delta$ and $Q$ all span the same dimension-$r$ subspace of $\mathbb{R}^{m}$. If $A$ is nonnegative, then $\Delta$ is contained in the positive orthant, so $\Delta\subseteq Q$. If $B$ is nonnegative then each column of $M$ is a conic combination of columns of $A$ with coefficients given by columns of $B$, hence $P\subseteq\Delta$. Conversely, one can construct a size-$r$ nonnegative factorization $(A,B)$ from a dimension-$r$ simplicial cone $\Delta$ that is nested between $P$ and $Q$ by taking the generating rays of $\Delta$ to be the columns of $A$. Therefore the matrix $M$ has nonnegative rank $r$ if and only if there exists a simplicial cone $\Delta$ such that $P\subseteq\Delta\subseteq Q$. Gillis and Glineur defined the restricted nonnegative rank of $M$ as the smallest number of rays of a cone that can be nested between $P$ and $Q$ [16], which is an upper bound on the nonnegative rank in the case that the rank and nonnegative rank differ.

The work of Vavasis [30] presents a second description of the same nested cones up to a linear transformation. Fix a particular rank factorization $(A,B)$ of $M$ (not necessarily nonnegative). All rank factorizations of $M$ have the form $(AC,C^{-1}B)$ where $C\in\mathbb{R}^{r\times r}$ is an invertible matrix. Let $P$ be the cone spanned by the columns of $B$; let $\Delta$ be the cone spanned by the columns of $C$; let $Q$ be the cone that is defined by $\{x\in\mathbb{R}^{r}:Ax\geq 0\}$. The linear map $A$ sends these three polyhedral cones to their counterparts in the first construction.

Zeros in a nonnegative factorization correspond to incidence relations between the three cones, $P$, $\Delta$ and $Q$. In particular, a zero in $A$ means that a ray of $\Delta$ lies on a facet of $Q$. A zero in $B$ means that a ray of $P$ lies on a facet of $\Delta$.

One often considers nested polytopes instead of nested cones. One gets nested polytopes from nested cones by intersecting the cones with an affine plane, which is usually defined by setting the sum of the coordinates to 1.

Below we present a different geometric picture to help understand when a rank-$r$ matrix has nonnegative rank $r$ and specifically when it lies on the boundary of the semialgebraic set. We will however at times refer to the nested polytopes $P\subseteq\Delta\subseteq Q$.

2.3 Nonnegative rank boundaries

Fixing $m$, $n$ and $r$, let $\mathbb{R}^{m\times n}$ denote the set of real $m\times n$ matrices, and $\mathbb{R}^{m\times n}_{\leq r}$ the subset with rank at most $r$. The set $\mathbb{R}^{m\times n}_{\leq r}$ is algebraic, meaning it is cut out by polynomial equations on the entries, namely by the $(r+1)\times(r+1)$ minors. There is an algebraic map

[TABLE]

given by matrix multiplication, and $\mathbb{R}^{m\times n}_{\leq r}$ is its image. Let $\mathcal{M}^{m\times n}_{\leq r}$ be the subset of $\mathbb{R}^{m\times n}_{\leq r}$ consisting of the matrices that also have nonnegative rank at most $r$. This set is the image of $\mu$ restricted to the $m\times r$ and $r\times n$ matrices with nonnegative entries. Certain combinations of these inequalities when mapped forward produce the polynomial inequalities that describe $\mathcal{M}^{m\times n}_{\leq r}$ as a subset of $\mathbb{R}^{m\times n}_{\leq r}$ (see Proposition 5.3). A set such as $\mathcal{M}^{m\times n}_{\leq r}$ that is described by a finite number of polynomial equations and inequalities is called a semialgebraic set. Its relative boundary has a finite number of (algebraic) boundary components, each where one of the defining inequalities attains equality. The boundary components are themselves irreducible semialgebraic sets, each of dimension one lower than $\mathcal{M}^{m\times n}_{\leq r}$. Some of the boundary components of $\mathcal{M}^{m\times n}_{\leq r}$ are straight-forward: for a matrix $M$ to have a nonnegative rank, each of its entries must be greater than or equal to zero. These inequalities define the trivial boundary components of $\mathcal{M}^{m\times n}_{\leq r}$.

Some boundary components of $\mathcal{M}^{m\times n}_{\leq r}$ consist of matrices that have infinitesimally rigid factorizations. Such factorizations are locally unique, so they are important for understanding which matrices have unique nonnegative factorizations. Using the ideas of rigidity theory, we show in Section 3 that infinitesimally rigid factorizations are characterized by certain patterns of zero entries in the factors. We give several necessary conditions on zero patterns that can result in infinitesimally rigid factorizations. These results generalize the previously known full characterization of such zero patterns for $r=3$ [18]. All boundary components of $\mathcal{M}^{m\times n}_{\leq 3}$ come from infinitesimally rigid factorizations, and there is only one zero pattern up to row and column permutation and transposition. For higher rank, characterizing these zero patterns is more complicated. In addition, we show in Sections 4.2 and 5.2 that for $r\geq 4$ there are other kinds of boundary components with no analogue in the rank 3 case, and some of these components do not lead to locally unique factorizations.

We will show in Section 5 that when a matrix $M$ lies in the relative interior of $\mathcal{M}^{m\times n}_{\leq r}$, the set of rank-$r$ nonnegative factorizations has the full dimension, so it is not uniquely decomposable. On the other hand, if $M$ is positive and lies on the relative boundary, then the nonnegativity constraints cut down the set of nonnegative factorizations to lower dimension. On some types of boundary components, the set of factorizations of $M$ is cut down to a single point, meaning the factorization is locally unique. Moreover if $M$ lies on no other boundary components, this factorization is globally unique. Understanding the boundary components of $\mathcal{M}^{m\times n}_{\leq r}$ then also provides an understanding of which matrices have unique nonnegative factorizations. The equations and inequalities describing the boundary components of each type provide semialgebraic conditions that can be checked on a matrix of rank $r$ to determine if it has a unique nonnegative rank-$r$ factorization.

3 Infinitesimally rigid factorizations

In this section, we will establish a connection between rigidity theory and nonnegative matrix factorizations. The setup is similar to the low-rank matrix completion case, although there are three main differences: The graph $G$ is always a complete bipartite graph, there are additional nonnegativity constraints, and the space of “trivial” deformations is much smaller. We will assume that nonnegative rank of a matrix is equal to its rank.

Let $G=(V,E)$ be the complete bipartite graph on $m+n$ vertices. As before, the vertices $V$ correspond to $a_{1},\ldots,a_{m},b_{1},\ldots,b_{n}$ and the edges $E$ correspond to the entries of a matrix $M$. We consider an infinitesimal motion of a framework parametrized by $t$. In addition to preserving the inner products $M_{ij}=a^{T}_{i}b_{j}$ for all $i\in[m],j\in[n]$, also $a_{i}$ and $b_{j}$ need to stay positive. Hence an infinitesimal motion has to satisfy

[TABLE]

for some $\epsilon>0$.

As before, let $A$ and $B$ be the rank-$r$ matrices with rows $a_{1}^{T},\ldots,a_{m}^{T}$ and columns $b_{1},\ldots,b_{n}$ respectively. Similarly, define $\dot{A}$ and $\dot{B}$ to be the matrices with rows $\dot{a}_{1}^{T},\ldots,\dot{a}_{m}^{T}$ and columns $\dot{b}_{1},\ldots,\dot{b}_{n}$ respectively. Then $M=AB$ and Equation (3.1) can be expressed as $A\dot{B}+\dot{A}B=0$. For the equation to hold, the column span of $\dot{A}$ must be contained in that of $A$ and similarly for the row spans of $\dot{B}$ and $B$. Therefore $\dot{A}=AD_{1}$ and $\dot{B}=-D_{2}B$ for $r\times r$ matrices $D_{1}$ and $D_{2}$. Moreover $-AD_{2}B+AD_{1}B=0$ and the fact that $A$ and $B$ are full rank implies that $D_{1}=D_{2}$. Therefore every solution to Equation (3.1) has the form $\dot{a}_{i}=D^{T}a_{i}$ and $\dot{b}_{j}=-Db_{j}$ with $D\in\mathbb{R}^{r\times r}$. Conversely it can be checked that any $a_{1},\ldots,a_{m},b_{1},\ldots,b_{n}$ with derivatives of this form satisfy Equation (3.1). The set of matrices $D\in\mathbb{R}^{r\times r}$ that define infinitesimal motions is

[TABLE]

If matrix $D$ is diagonal then $\dot{a}_{i}=D^{T}a_{i}$ and $\dot{b}_{j}=-Db_{j}$ always define an infinitesimal motion, and such a motion is called trivial.

Definition 3.1**.**

A framework is infinitesimally rigid if all its infinitesimal motions are trivial.

An infinitesimal motion does not necessarily correspond to any actual smooth path through $(A,B)$ in the space of nonnegative factorizations of $M$, but only to a tangent direction that does not violate nonnegativity. Thus infinitesimal rigidity is not a necessary (and also not a sufficient) condition for the uniqueness of a nonnegative matrix factorization. However, every infinitesimally rigid nonnegative factorization is locally rigid (Proposition 4.2) and local rigidity is a necessary condition for the uniqueness of a nonnegative matrix factorization. In fact, when Kruskal rank of a certain matrix is maximal possible, then a locally rigid nonnegative factorization is infinitesimally rigid (Proposition 4.8). These results allow us to state in Section 4 so far the strongest necessary condition for the uniqueness of a nonnegative factorization.

Example 3.2**.**

A rank-3 matrix $M$ with positive entries is on the boundary of $\mathcal{M}_{3}^{m\times n}$ if and only if all nonnegative factorizations of $M$ are infinitesimally rigid. This follows from the analysis of Mond, Smith and van Straten in [23, Lemma 4.3]. One can show that a size-3 infinitesimally rigid nonnegative factorization has up to permuting rows of $A$, permuting columns of $B$, simultaneously permuting columns of $A$ and rows of $B$, and switching $A$ and $B^{T}$ the following form

[TABLE]

We will now study the set $W_{(A,B)}$ of matrices $D\in\mathbb{R}^{r\times r}$ that define infinitesimal motions. The inequality $a_{i}+tD^{T}a_{i}\geq 0$ is trivially satisfied for $t\in[0,\epsilon)$ and some $\epsilon>0$ for all positive coordinates of $a_{i}$. Hence a row $a^{T}_{i}$ of $A$ defines an inequality on the $j$th column of $D$ if and only if the $j$th coordinate of $a_{i}$ is zero. The corresponding inequality is $d^{T}_{j}a_{i}\geq 0$ where $d_{j}$ denotes the $j$th column of $D$. For each $i=1,\ldots,m$, let $S_{i}\subseteq\{1,\ldots,r\}$ be the set of entries of $a_{i}$ that are zero. Then $D\in W_{(A,B)}$ satisfies $d^{T}_{j}a_{i}\geq 0$ for all $j\in S_{i}$. Equivalently $\langle a_{i}e^{T}_{j},D\rangle\geq 0$, where $\langle\cdot,\cdot\rangle$ denotes entry-wise inner product on $r\times r$ matrices.

On the other hand, a column $b_{i}$ of $B$ defines an inequality on the $j$th row of $-D$ if and only if the $j$th coordinate of $b_{i}$ is zero. This inequality is $-d^{\prime}_{j}b_{i}\geq 0$ where $d^{\prime}_{j}$ denotes the $j$th row of $D$. For each $i=1,\ldots,n$, let $T_{i}\subseteq\{1,\ldots,r\}$ be the set of entries of $b_{i}$ that are zero. Then $D\in W_{(A,B)}$ satisfies $-d^{\prime}_{j}b_{i}\geq 0$ or equivalently $\langle-e_{j}b^{T}_{i},D\rangle\geq 0$ for $j\in T_{i}$.

Hence $W_{(A,B)}$ is a polyhedral cone and we have described it in terms of its facet inequalities, but it will often be easier to work with its dual cone. For each $i=1,\ldots,m$, define $\mathcal{A}_{i}=\{a_{i}e^{T}_{j}\mid j\in S_{i}\}$, and for each $i=1,\ldots,n$, define $\mathcal{B}_{i}=\{-e_{j}b^{T}_{i}\mid j\in T_{i}\}$. Then

[TABLE]

Proposition 3.3**.**

A nonnegative factorization $(A,B)$ is infinitesimally rigid if and only if $W_{(A,B)}^{\vee}$ is isomorphic to $\mathbb{R}^{r^{2}-r}$ (meaning $W_{(A,B)}^{\vee}$ is an $(r^{2}-r)$-dimensional real vector space).

Proof.

If the cone $W_{(A,B)}$ consists only of $r\times r$ diagonal matrices then the dual cone $W_{(A,B)}^{\vee}$ consists of all $r\times r$ matrices that are zero along the diagonal. This is a linear space of dimension $r^{2}-r$. Conversely, if $W_{(A,B)}$ contains other matrices, then its dimension is strictly larger than $r$. Hence the dimension of the largest subspace contained in $W_{(A,B)}^{\vee}$ is strictly less than $r^{2}-r$. ∎

Proposition 3.3 gives an algorithm for checking whether a nonnegative factorization is infinitesimally rigid. For example, open source tool Normaliz [3] allows one to compute the largest linear subspace contained in a cone given by its extremal rays. However, Proposition 3.3 does not give insight how to construct infinitesimally rigid nonnegative matrix factorizations. To solve this problem, we give a completely combinatorial necessary condition for a nonnegative matrix factorization to be infinitesimally rigid. In Appendix A, we will use this result to construct infinitesimally rigid nonnegative matrix factorizations for $5\times 5$ matrices of nonnegative rank four, which is the first nontrivial case.

Theorem 3.4**.**

If $(A,B)$ is an infinitesimally rigid nonnegative rank-$r$ factorization then

• •

$A$* and $B$ have at least $r^{2}-r+1$ zeros in total and*

• •

for every distinct pair $i,j$ taken from $1,\ldots,r$, there must be a row of $A$ with a zero in position $i$ and not in position $j$. Similarly for the columns of $B$.

Proof.

A nonnegative factorization $(A,B)$ being infinitesimally rigid is equivalent to $W_{(A,B)}^{\vee}\cong\mathbb{R}^{r^{2}-r}$. To express $\mathbb{R}^{r^{2}-r}$ as the convex cone of a finite number of vectors requires at least $r^{2}-r+1$ vectors. The size of the generating set defining $W_{(A,B)}^{\vee}$ is equal to the total number of zeros in $A$ and $B$.

The vectors coming from $A$ are nonnegative and the ones from $B$ are nonpositive. If $W_{(A,B)}^{\vee}\cong\mathbb{R}^{r^{2}-r}$, for each coordinate there must be at least one vector with a strictly positive value there, and one with a strictly negative value. To get a positive value in coordinate $d_{ij}$ requires $A$ to have a row with zero in the $j$th entry and a non-zero value in the $i$th entry. Similarly for columns of $B$. ∎

The second condition is a necessary condition for the uniqueness of nonnegative matrix factorization, see [19, 15].

Example 3.5**.**

Let $r=3$. The zero pattern (3.3) is the unique zero pattern with seven zeros that fulfills the conditions in Theorem 3.4, up to allowed permutations.

We conclude this section with some properties of infinitesimally rigid nonnegative factorizations.

Corollary 3.6**.**

If $(A,B)$ is an infinitesimally rigid nonnegative rank-$r$ factorization with exactly $r^{2}-r+1$ zeros, then $AB$ is strictly positive.

Proof.

If $(A,B)$ is infinitesimally rigid, then the dual cone $W^{\vee}_{(A,B)}$ is equal to the space of matrices with zero diagonal of dimension $r^{2}-r$. The zeros of $A$ and $B$ correspond to the elements of a distinguished generating set of $W^{\vee}_{(A,B)}$ as described above. A generating set of size $r^{2}-r+1$ is minimal, so the only linear relation among the generators must be among all $r^{2}-r+1$.

If $AB$ has a zero in entry $ij$ then row $a_{i}$ of $A$ and column $b_{j}$ of $B$ have zeros in complementary positions so that $a_{i}\cdot b_{j}=0$. Since the support of $b_{j}$ is contained in the set of columns for which $a_{i}$ is zero, the outer product matrix $a_{i}^{T}b_{j}^{T}$ can be expressed as a nonnegative combination of the dual vectors coming from $a_{i}$. Similarly, the matrix $-a_{i}^{T}b_{j}^{T}$ can be expressed as a nonnegative combination of the dual vectors coming from $b_{j}$. Summing these gives a linear relation among a strict subset of the generators, which is a contradiction. ∎

Corollary 3.7**.**

If $(A,B)$ is an infinitesimally rigid nonnegative factorization, then there is at least one zero in every column of $A$ and in every row of $B$.

Proof.

It follows directly from Theorem 3.4. ∎

Corollary 3.8**.**

If $M$ is strictly positive and $(A,B)$ is an infinitesimally rigid nonnegative rank-$r$ factorization of $M$, then there are at most $r-2$ zeros in every row of $A$ and in every column of $B$.

Proof.

Since $M$ is positive, no row of $A$ or column of $B$ can contain only zeros. If a row of $A$ contains $r-1$ zeros, then there has to be a row of $B$ that does not contain any zero, because otherwise $AB$ would have a zero entry. This contradicts Corollary 3.7. ∎

Lemma 3.9**.**

If $(A,B)$ is an infinitesimally rigid nonnegative rank-$r$ factorization with $r^{2}-r+1$ zeros, then there are at most $r-1$ zeros in every column of $A$ and in every row of $B$.

Proof.

As in the proof of Corollary 3.6, the only linear relation among the generators of $W^{\vee}_{(A,B)}$ must be among all $r^{2}-r+1$ generators. If there were $r$ zeros in the same column of $A$, then there would be $r$ generators of $W^{\vee}_{(A,B)}$ contained in a $r-1$ dimensional subspace, implying a smaller linear relation which is impossible. Similarly for the case of $r$ zeros in a row of $B$. ∎

This argument can be generalized to forbid other configurations of zeros that concentrate too many generators of $W^{\vee}_{(A,B)}$ into too small a support.

Lemma 3.10**.**

Let $(A,B)$ be an infinitesimally rigid nonnegative rank-$r$ factorization with $r^{2}-r+1$ zeros. Let $\alpha,\beta\subseteq[r]$ and suppose $A$ has a $k\times|\alpha|$ submatrix of zeros with columns $\alpha$, and $B$ has a $|\beta|\times\ell$ submatrix of zeros with rows $\beta$. Then

[TABLE]

Proof.

As in the proof of Corollary 3.6, a generating set of size $r^{2}-r+1$ is minimal, so the only linear relation among the generators must be among all $r^{2}-r+1$. It can be checked that the zeros of $A$ described above correspond to $k|\alpha|$ generators of $W^{\vee}_{(A,B)}$ supported on entries $([r]\setminus\alpha)\times\alpha$. Similarly the zeros of $B$ corresponds to $\ell|\beta|$ generators supported on entries $\beta\times([r]\setminus\beta)$. The intersection of these two supports is $(\beta\setminus\alpha)\times(\alpha\setminus\beta)$. The number of generators cannot exceed the number of entries they are supported on, which gives the inequality. ∎

Lemma 3.9 is the special case when $\alpha$ is a singleton and $\beta$ is empty or the reverse, and this case seems to be the most applicable condition when $r$ is small.

4 Locally rigid factorizations

4.1 Definition and properties

Definition 4.1**.**

A nonnegative factorization $(A,B)$ is locally rigid if all nonnegative factorizations of $AB$ in a neighborhood of $(A,B)$ are obtained by scaling the columns of $A$ and rows of $B$.

If a matrix has a unique size-$r$ nonnegative factorization, then this factorization has to be locally rigid. We recall that the second condition in Theorem 3.4 is a necessary condition for the uniqueness of a nonnegative matrix factorization by [19, Theorem 3]. In fact, it is a necessary condition for local rigidity of a nonnegative matrix factorization using the argument in [15, Remark 7].

It concludes from the definition of an infinitesimally rigid nonnegative factorization that all nonnegative factorizations in some neighborhood are obtained from scalings.

Proposition 4.2**.**

If nonnegative factorization $(A,B)$ is infinitesimally rigid, then it is locally rigid.

We will see in the next subsection that the converse is true if a certain matrix achieves its maximal possible Kruskal rank.

Example 4.3**.**

It follows from the discussion in Example 3.2 that if $M$ lies on the boundary of $\mathcal{M}_{3}^{m\times n}$, then all its nonnegative factorizations are locally rigid. This can be also seen using the geometric characterization of boundaries in [18, Corollary 4.4]. Namely, a matrix with positive entries lies on the boundary of $\mathcal{M}_{3}^{m\times n}$ if and only if for every nonnegative factorization of the matrix the corresponding geometric configuration satisfies that (i) every vertex of the intermediate triangle lies on an edge of the outer polygon, (ii) every edge of the intermediate triangle contains a vertex of the inner polygon, (iii) a vertex of the intermediate triangle coincides with a vertex of the outer polygon or an edge of the intermediate triangle contains an edge of the inner polygon. Such geometric configurations are isolated for fixed inner and outer polygons, hence the corresponding nonnegative factorizations are locally rigid.

In the rest of the subsection, we will explore modifications of locally rigid nonnegative matrix factorizations.

Lemma 4.4**.**

Let $(A,B)$ be a locally rigid factorization. Let $(A^{\prime},B^{\prime})$ be a factorization that is obtained from $(A,B)$ by erasing all rows of $A$ and columns of $B$ that do not contain any zero entries. Then $(A^{\prime},B^{\prime})$ is locally rigid.

We will postpone proof of Lemma 4.4 until Section 5.1 where we take a more geometric view on rigidity.

Lemma 4.5**.**

Let $(A,B)$ be a nonnegative factorization. For $\varepsilon>0$ small enough, there exists $A^{\prime}$ obtained from $A$ by adding at most $r$ strictly positive rows and $B^{\prime}$ obtained from $B$ by adding at most $r$ strictly positive columns such that any nonnegative factorization of $A^{\prime}B^{\prime}$ is in the $\varepsilon$-neighborhood of $(A^{\prime}P,P^{-1}B^{\prime})$ for some $r\times r$ scaled permutation matrix $P$.

Proof.

Consider the geometric configuration of cones in $\mathbb{R}^{r}$ corresponding to the factorization $(A,B)$. Since $(A,B)$ is a nonnegative factorization, the intermediate cone is spanned by the unit vectors. We add $r$ strictly positive rows to $A$ that correspond to hyperplanes at most distance $\delta$ from the facets of the intermediate cone. We add $r$ strictly positive columns to $B$ that correspond to points that are at most distance $\delta$ from the vertices of the intermediate cone. Neither of these operations changes incidence relations between the three cones. The new outer cone is contained in $(1+\delta)$ times larger copy of the intermediate cone and the new inner cone contains a $(1-\delta)$ times smaller copy of the intermediate cone. For $\varepsilon$ small enough, there exists $\delta$ such that the only other cones with $r$ rays that one can be nested between a larger and smaller copy of the intermediate cone give factorizations that are in the $\varepsilon$-neighborhood of $(A^{\prime}P,P^{-1}B^{\prime})$. ∎

Definition 4.6**.**

A nonnegative factorization $(A,B)$ is globally rigid if all nonnegative factorizations of $AB$ are obtained by scaling and permuting the columns of $A$ and rows of $B$.

Corollary 4.7**.**

Given a locally rigid nonnegative factorization $(A,B)$, then by adding at most $r$ strictly positive rows to $A$ and at most $r$ strictly positive columns to $B$, one can get a globally rigid nonnegative matrix factorization.

4.2 When is infinitesimal rigidity equivalent to local rigidity?

Let $Z_{(A,B)}$ be a matrix with columns equal to the elements of $\mathcal{A}_{1}\cup\cdots\cup\mathcal{A}_{m}\cup\mathcal{B}_{1}\cup\cdots\cup\mathcal{B}_{n}$. Let $c$ be the number of columns of $Z_{(A,B)}$. Let the Kruskal rank be the maximal value $k$ such that any $k$ columns are linearly independent. We denote the Kruskal rank of $Z_{(A,B)}$ by $\mathrm{K\text{-}rank}(Z_{(A,B)})$. We will show that if $\mathrm{K\text{-}rank}(Z_{(A,B)})=\min(c,r^{2}-r)$, then local rigidity implies infinitesimal rigidity. This result can be seen as an adaptation of Theorem 2.1 by Asimow and Roth to nonnegative matrix factorizations.

Proposition 4.8**.**

If $(A,B)$ is a nonnegative factorization that is locally rigid but not infinitesimally rigid, then $\mathrm{K\text{-}rank}(Z_{(A,B)})<\min(c,r^{2}-r)$.

Proof.

We assume that $(A,B)$ is nonnegative factorization that is locally rigid but not infinitesimally rigid. We will show that $r<\dim W_{(A,B)}<r^{2}$. The first inequality follows immediately from the fact that $(A,B)$ is not infinitesimally rigid. The second inequality follows from the fact that $(A,B)$ is locally rigid by applying either Proposition 5.6, or the following argument that does not require the machinery of Section 5.

Since $(A,B)$ is not infinitesimally rigid there exists $D\in W_{(A,B)}$ that is not diagonal. If $(\dot{a}_{i})_{j}=(D^{T}a_{i})_{j}$ is strictly positive for all $(i,j)$ such that $(a_{i})_{j}$ is zero and $(\dot{b}_{j})_{i}=(-Db_{j})_{i}$ is strictly positive for all $(i,j)$ such that $(b_{j})_{i}$ is zero, then the corresponding motion gives nonnegative factorizations for all $t\in[0,\epsilon)$ for some $\epsilon$ small enough. Hence a necessary condition for a locally rigid nonnegative factorization that is not infinitesimally rigid is that $(\dot{a}_{i})_{j}=(D^{T}a_{i})_{j}=0$ for some $(i,j)$ such that $(a_{i})_{j}=0$ or $(\dot{b}_{j})_{i}=(-Db_{j})_{i}=0$ for some $(i,j)$ such that $(b_{j})_{i}=0$. Moreover, there exists at least one pair $(i,j)$ such that for all $D\in W_{(A,B)}$ we have $(a_{i})_{j}=(D^{T}a_{i})_{j}=0$ or $(b_{j})_{i}=(-Db_{j})_{i}=0$, because otherwise one could take a conic combination of matrices $D$ with $(D^{T}a_{i})_{j}=0$ and $(-Db_{j})_{i}=0$ for different $(i,j)$ to get an element of $W_{(A,B)}$ with no $(D^{T}a_{i})_{j}=0$ or $(-Db_{j})_{i}=0$.

Without loss of generality we assume that $(a_{i})_{j}=(D^{T}a_{i})_{j}=0$ for all $D\in W_{(A,B)}$. Hence $a_{i}e^{T}_{j}$ and $-a_{i}e^{T}_{j}$ both belong to the dual cone $W^{\vee}_{(A,B)}$. Since the dual cone has a non-trivial lineality space, $\dim W_{(A,B)}<r^{2}$.

From the fact that $r<\dim W_{(A,B)}<r^{2}$, it follows that the dual cone, $W^{\vee}_{(A,B)}$, has dimension-$k$ lineality space with $0<k<r^{2}-r$. A generating set of $W^{\vee}_{(A,B)}$ has a subset of size at least $k+1$ that generates the lineality space, and any $k+1$ of those generators are linearly dependent. Therefore $Z_{(A,B)}$ has $k+1$ columns that are linearly dependent, so $\mathrm{K\text{-}rank}(Z_{(A,B)})\leq k<r^{2}-r$. Because $k+1\leq c$, this also implies $\mathrm{K\text{-}rank}(Z_{(A,B)})<c$. ∎

Corollary 4.9**.**

If a nonnegative factorization $(A,B)$ is locally rigid, then $W^{V}_{(A,B)}\cong\mathbb{R}^{r^{2}-r}$ or $\mathrm{K\text{-}rank}(Z_{(A,B)})<\min(c,r^{2}-r)$.

Since local rigidity is a necessary condition for global rigidity, the conditions in Corollary 4.9 are necessary for the uniqueness of a nonnegative factorization. We will also state Corollary 4.10 that is a simplified version of Corollary 4.9. Corollary 4.10 directly strengthens the necessary condition for uniqueness in [19, Theorem 3] that states that the support of any column of $A$ cannot be contained in the support of any other column of $A$ and the support of any row of $B$ cannot be contained in the support of any other row of $B$.

Corollary 4.10**.**

If $(A,B)$ is a globally rigid nonnegative factorization, then the support of any column of $A$ cannot be contained in the support of any other column of $A$, the support of any row of $B$ cannot be contained in the support of any other row of $B$, and the matrices $A$ and $B$ have at least $r^{2}-r+1$ zeros in total or $\mathrm{K\text{-}rank}(Z_{(A,B)})<\min(c,r^{2}-r)$.

Separability based sufficient conditions for uniqueness, e.g. in [10] and [19], satisfy the additional condition that $A$ and $B$ have at least $r^{2}-r+1$ zeros in total, because the separability condition quaratees that one of the factors has at least $r^{2}-r$ zeros and there is at least one additional zero coming from the zero pattern in the other factor. It is unknown which of the two additional conditions is satisfied by sufficiently scattered based sufficient conditions, discussed in [12]. Our methods do not compare directly with methods that guarantee identifiability under further assumptions such as orthogonality of a factor, maximal sparseness, volume minimization or maximization of the polytope associated to one of the factors.

Corollary 4.9 together with the necessary condition for uniqueness from [19, Theorem 3] gives Algorithm 1 for determining infinitesimal and local rigidity of a nonnegative matrix factorization.

To test global rigidity of a size-$r$ nonnegative matrix factorization $(A,B)$, one can run a program that searches numerically for size-$r$ nonnegative matrix factorizations of the matrix $AB$. If $(A,B)$ is not globally rigid, then we do not expect the program to output precisely $(A,B)$ up to permutations and scalings. On the contrary, if the program outputs only $(A,B)$ up to permutations and scalings over multiple runs, then this provides evidence towards $(A,B)$ being globally rigid. This approach is further discussed in Appendix A.

In the rest of the section, we present a locally rigid factorization which is not infinitesimally rigid. The example we present is a modification of an example by Shitov [25] that he uses to show that nonnegative rank depends on the field. His example is a matrix of nonnegative rank five, we present a geometric configuration corresponding to a matrix of nonnegative rank four. Checking local rigidity involves studying signs of second derivatives in addition to the requirements on zeros and first derivatives.

Example 4.11**.**

The outer polytope $Q=\text{conv}(\Omega_{1},\Omega_{2},A_{i},B_{i},C_{i}:1\leq i\leq 3)$ is a modification of a simplex. Let $\varepsilon=1/20$. Three vertices of this simplex are replaced by small triangles conv$(A_{i},B_{i},C_{i})$, where

[TABLE]

The last vertex of the simplex is replaced by a small edge conv$(\Omega_{1},\Omega_{2})$. The vertices $\Omega_{1}$ and $\Omega_{2}$ are points on the line

[TABLE]

that are sufficiently close to and on the opposite sides of $(1/3,1/3,1/3,0)$. For example, one can take $t$ to be equal to $1/40$ and $-1/40$. Here $0.416827$ is an approximate number and we will explain later how to get the exact value.

The intermediate simplex $\Delta$ is conv$(\Omega,V_{1},V_{2},V_{3})$, where

[TABLE]

The vertex $\Omega$ lies on the edge conv$(\Omega_{1},\Omega_{2})$ of the outer polytope. All other vertices $V_{i}$ lie on the triangles conv$(A_{i},B_{i},C_{i})$.

The inner polytope $P$ is conv$(W,W_{i},F_{ij},H:1\leq i\leq 3,1\leq j\leq 2)$, where

[TABLE]

It has one vertex close to every vertex of the intermediate simplex: The vertex $W$ is close to $\Omega$ and the vertices $W_{i}$ are close to $V_{i}$. Moreover, there are two vertices on each facet of the intermediate simplex besides the facet that is opposite to $\Omega$: The vertices $F_{ij}$ lie on the facet of the simplex spanned by all vertices but $V_{i}$. The interior polytope also contains the vertex $H$ that lies on the facet of the intermediate simplex that is opposite to $\Omega$.

The pairwise inclusions of the three polytopes are depicted in Figure 1. The matrix $M$ corresponding to this geometric configuration is obtained by evaluating the facets of the outer polytope $Q$ at the vertices of the inner polytope $P$. The facets of $Q$ can be found for example using polymake [14]. The matrix $A$ in the nonnegative factorization is obtained by evaluating the facets of $Q$ at the vertices of $B$; the matrix $B$ is obtained by evaluating the facets of $Q$ at the vertices of $P$. The nonnegative factorization has the following zero pattern (after removing rows of $A$ and columns of $B$ that do not contain zeros):

[TABLE]

The number of zeros in this factorization is $12$, so this factorization is not infinitesimally rigid. We will show that it is locally rigid, i.e. that $\Delta$ is the only simplex that can be nested between $P$ and $Q$. The proof is analogous to the proof in [25]. We present it here so that the reader is able to directly check the correctness of our example.

Since $P$ and $Q$ are constructed such that they are close to $\Delta$, any other simplex $\Delta^{\prime}$ that can be nested between $P$ and $Q$ must be close to $\Delta$. We will give a parametrization of simplices and show that any simplex $\Delta^{\prime}$ close to $\Delta$ can be parametrized in such a way.

We restrict to the affine plane in $\mathbb{R}^{4}$ defined by $x_{1}+x_{2}+x_{3}+x_{4}=1$. Let $\omega$ be a point on this plane such that $\|\Omega-\omega\|<\varepsilon$. Let $f_{ij}=(F_{ij}-(0,0,0,v_{ij}))/(1-v_{ij})$ where $v_{ij}$ are parameters. Let $H_{i}(\omega,v)$ be the hyperplane through $f_{i1},f_{i2},\omega$. Define the point $V_{i}(\omega,v)$ as the intersection of the hyperplanes $x_{i}=0$ and $H_{j}(\omega,v)$ for $j\neq i$. Define $\Delta(\omega,v)=\text{conv}(V_{1}(\omega,v),V_{2}(\omega,v),V_{3}(\omega,v),\omega)$. Then $\Delta=\Delta(\Omega,0)$.

Since $\Delta^{\prime}$ is close to $\Delta$, the facet of $\Delta^{\prime}$ opposite to the vertex $V^{\prime}_{i}$ intersects the line of $f_{i1}$’s and the line of $f_{i2}$’s. Moreover, the points where the facet of $\Delta^{\prime}$ intersects these lines correspond to nonnegative $v_{ij}$, because $F_{ij}\in P\subset\Delta^{\prime}$ correspond to zero parameters and going outwards from $P$ on the line of $f_{ij}$’s increases the value of the parameters $v_{ij}$. Furthermore, since maximal simplices inside $Q$ have vertices on the boundary of $Q$, we can assume that this is the case for $\Delta^{\prime}$ and hence $\Delta^{\prime}=\Delta(\omega,v)$ for some $\omega\in Q$ and $v\geq 0$.

Let $\Psi(\omega,v)=\det(V_{1}(\omega,v),V_{2}(\omega,v),V_{3}(\omega,v),H)$. We note that $\Psi(\Omega,0)=0$ and $\det(V_{1},V_{2},V_{3},\Omega)>0$. To show that $\Delta$ is the only simplex that can be nested between $P$ and $Q$ it is enough to show that for all other $\Delta(\omega,v)$ close to $\Delta$ with $\omega\in Q$ and $v\geq 0$, we have $H\not\in\Delta(\omega,v)$. This is equivalent to $\Psi(\omega,v)<0$ and $(\Omega,0)$ being a local maximum of $\Psi$ when $\omega\in Q$ and $v\geq 0$. It can be checked that the partial derivatives $\partial\Psi/\partial v_{ij}$ and the directional derivatives in the directions from $(\Omega,0)$ to $(A_{i},0),(B_{i},0),(C_{i},0)$ are negative at $(\Omega,0)$. Finally, on the line

[TABLE]

we have $\Psi^{\prime}=0$ and $\Psi^{\prime\prime}<0$. In fact, the number $0.416827$ is an approximation of the solution for $x$ in the equation $\frac{\partial\Psi((1/3,1/3-2t,1/3+t,xt),v)}{\partial t}|_{(t=0,v=0)}=0$.

This example is a modification of an infinitesimally rigid example with $13$ zeros where a vertex of the outer polytope is replaced with an edge $\text{conv}(\Omega_{1},\Omega_{2})$. The corresponding nonnegative factorization would have an extra row in $A$ with zero in the last column. The vertex of the intermediate simplex that for the infinitesimally rigid configuration coincides with the vertex of the outer polytope lies now on the new edge. The only difference between the two examples is that theoretically one can now move the vertex of the intermediate simplex also along the edge $\text{conv}(\Omega_{1},\Omega_{2})$, but in fact this is not possible, because the local maximum of $\Psi$ on $\text{conv}(\Omega_{1},\Omega_{2})$ is $\Omega$. By results of Mond, Smith and van Straten [23], it is not possible to construct an analogous example for polygons.

5 Rigidity and boundaries

In this section we use $\mathcal{M}^{m\times n}_{r}$ to denote the set of $m\times n$ matrices with rank and nonnegative rank both equal to exactly $r$. A matrix of nonnegative rank three is on the boundary of $\mathcal{M}_{3}^{m\times n}$ if and only if it has a zero entry or all its nonnegative factorizations are infinitesimally rigid. The goal of this section is to study the connection between boundaries of $\mathcal{M}_{r}^{m\times n}$ and rigidity theory for $r\geq 4$. We already saw in Example 4.11 that there exist locally rigid nonnegative matrix factorizations that are not infinitesimally rigid. Combining this with results in Section 5.1, one can show that there exists a matrix on the boundary of $\mathcal{M}_{4}^{m\times n}$ for $m,n$ large enough that has a locally rigid nonnegative factorization but no infinitesimally rigid nonnegative factorizations. Furthermore, in Section 5.2 we will show that there exist strictly positive matrices on the boundary of $\mathcal{M}_{r}^{m\times n}$ for $r\geq 4$ that have nonnegative factorizations that are not locally rigid. There exists a neighborhood of a such factorization whose dimension is strictly between $r$ and $r^{2}$, the minimal and maximal dimensions of spaces of factorizations.

5.1 Geometry of nonnegative matrix factorizations

As in Section 2.3 let $\mu$ be the usual matrix multiplication map, but we now restrict the domain to pairs of matrices with full rank $r$:

[TABLE]

The image of $\mu$ is $\mathbb{R}_{r}^{m\times n}$, the set of $m\times n$ matrices with rank $r$. The positive orthant $(\mathbb{R}_{r}^{m\times r})_{\geq 0}\times(\mathbb{R}_{r}^{r\times n})_{\geq 0}$ is mapped onto $\mathcal{M}_{r}^{m\times n}$, the set of rank-$r$ matrices with nonnegative rank $r$. The trivial boundary of $\mathcal{M}_{r}^{m\times n}$ consists of such matrices with at least one zero entry.

Fix a rank-$r$ matrix $M$ with strictly positive entries and a rank factorization $(A,B)$ with $M=AB$. The set of all rank factorizations of $M$ is the fiber

[TABLE]

This set is a real $r^{2}$-dimensional smooth irreducible variety. Let

[TABLE]

$F$ is the graph of the inverse function on $r\times r$ matrices. The injective linear map

[TABLE]

[TABLE]

sends $F$ to $\mu^{-1}(M)$. The image of $\nu_{(A,B)}$ is the subspace of pairs $(\alpha,\beta)$ such that the columns of $\alpha$ are in the columns span of $A$ and the rows of $\beta$ are in the row span of $B$.

Proposition 5.1**.**

The map $\mu:\mathbb{R}_{r}^{m\times r}\times\mathbb{R}_{r}^{r\times n}\to\mathbb{R}_{r}^{m\times n}$ is a fiber bundle, with fiber $F$.

Proof.

A matrix $M\in\mathbb{R}^{m\times n}_{r}$ has a set of $r$ linearly independent columns. Given a rank factorization $(A,B)$ of $M$, the same set of columns is linearly independent in $B$. Call the $r\times r$ submatrix they form $C$. Then $(AC,C^{-1}B)$ is a rank factorization of $M$ with $C^{-1}B$ having the $r\times r$ submatrix in these columns equal to the identity, and this is the unique factorization of $M$ with that property. Let $K$ be the subset of $\mathbb{R}^{m\times r}_{r}\times\mathbb{R}^{r\times n}_{r}$ of pairs $(\alpha,\beta)$ in which $\beta$ has this particular submatrix equal to the identity.

All matrices in $\mathbb{R}^{m\times n}_{r}$ have a unique factorization in $K$ unless there is linear dependence among the chosen columns. Such exceptions form a lower dimensional subset, so in particular $M$ has a neighborhood $X$ of matrices with factorizations in $K$. Then $\mu^{-1}(X)$ has product structure $(\mu^{-1}(X)\cap K)\times F$ by map

[TABLE]

which can be checked is continuous with continuous inverse. This proves the fiber bundle structure of $\mu$. ∎

A factorization $(AC,C^{-1}B)$ of $M$ is a nonnegative factorization of $M$ if $AC\in(\mathbb{R}_{r}^{m\times r})_{\geq 0}$ and $C^{-1}B\in(\mathbb{R}_{r}^{r\times n})_{\geq 0}$. Let $c_{1},\ldots,c_{r}$ denote the columns of $C$ and $c^{\prime}_{1},\ldots,c^{\prime}_{r}$ the rows of $C^{-1}$. The inequality $Ac_{i}\geq 0$ gives $m$ linear inequalities on $c_{i}$ and defines a polyhedral cone in $\mathbb{R}^{r}$ with at most $m$ facets which we will denote $P_{A}$. Similarly $c^{\prime}_{i}B\geq 0$ defines a polyhedral cone $P_{B^{T}}$ in $(\mathbb{R}^{r})^{*}$ with at most $n$ facets. The nonnegative factorizations of $M$ then correspond to the set $F\cap(P_{A}^{\times r}\times P_{B^{T}}^{\times r})$. Let $U_{(A,B)}:=(P_{A}^{\times r}\times P_{B^{T}}^{\times r})$, which is itself a polyhedral cone.

Fixing $M$ and a rank factorization $(A,B)$, the injective linear map $\nu_{(A,B)}$ that sends $F$ to $\mu^{-1}(M)$ also maps cone $U_{(A,B)}$ to $(\mathbb{R}_{r}^{m\times r})_{\geq 0}\times(\mathbb{R}_{r}^{r\times n})_{\geq 0}\cap\operatorname{im}(\nu_{(A,B)})$. The boundary of $U_{(A,B)}$ maps to pairs of matrices that have at least one zero entry. Because $M$ is assumed to have positive entries, $\operatorname{im}(\nu_{(A,B)})$ is not contained in a coordinate hyperplane of $\mathbb{R}_{r}^{m\times r})\times\mathbb{R}_{r}^{r\times n})$. Therefore the interior of $U_{(A,B)}$ maps to pairs of matrices with positive entries. Sometimes it will be convenient to work in one or the other system of coordinates.

Remark 5.2**.**

$P_{A}$ is the outer cone, $Q$, and $P_{B^{T}}$ is dual to the inner cone, $P$, in the second geometric characterization in Section 2.2.

If $(A,B)$ is a nonnegative factorization of $M$ then $(AD,D^{-1}B)$ is as well for any diagonal matrix $D$ with positive diagonal entries. We will generally be interested only in factorizations modulo this scaling.

Now we have introduced the tools for proving Lemma 4.4.

Proof of Lemma 4.4.

Let $(A,B)$ be a locally rigid factorization. Let $(A^{\prime},B^{\prime})$ be a factorization that is obtained from $(A,B)$ by erasing all rows of $A$ and columns of $B$ that do not contain any zero entries.

For the sake of contradiction, assume that $(A^{\prime},B^{\prime})$ is not locally rigid. Equivalently every neighborhood of $(I,I)$ in $F\cap U_{(A^{\prime},B^{\prime})}$ contains a pair $(C,C^{-1})$ where $C$ is not diagonal. This implies that there is a row $a_{i}$ of $A$ with positive entries and a column $c_{j}$ of $C$ such that $a_{i}c_{j}<0$ or there is a column $b_{i}$ of $B$ with positive entries and a row $c^{\prime}_{j}$ of $C^{-1}$ such that $c^{\prime}_{j}b_{i}<0$. Let $c_{\max}$ be the maximal entry of $A$ and $B$; let $c_{\min}$ be the minimal non-zero entry of $A$ and $B$. Consider the $\varepsilon$-neighborhood of $(I,I)$ where $\varepsilon=\frac{c_{\min}}{c_{\min}+(r-1)c_{\max}}$. For any $(C,C^{-1})$ in this neighborhood, every non-diagonal entry of $C$ is greater than $-\varepsilon$ and every diagonal entry is greater than $1-\varepsilon$. Since $A$ and $B$ are nonnegative, we have $a_{i}c_{j}\geq-(r-1)\varepsilon c_{\max}+(1-\varepsilon)c_{min}=0$ and similarly $c^{\prime}_{j}b_{i}\geq 0$ for all $i,j$. ∎

Proposition 5.3**.**

Positive $M\in\mathcal{M}_{r}^{m\times n}$ lies on boundary of $\mathcal{M}_{r}^{m\times n}$ if and only if every nonnegative factorization $(A,B)$ of $M$ has at least one zero entry.

Proof.

Suppose $M$ has a strictly positive rank factorization $(A,B)$. Then $(A,B)$ has a relatively open neighborhood $W$ contained in $\mu^{-1}(M)\cap(\mathbb{R}_{r}^{m\times r})_{>0}\times(\mathbb{R}_{r}^{r\times n})_{>0}$. Since $\mu$ is a fiber bundle, it is an open mapping. Therefore $\mu(W)$ is an open neighborhood of $M$ in $\mathcal{M}_{r}^{m\times n}$, so $M$ is in the interior.

Suppose $M$ does not have any strictly positive rank factorizations. Equivalently $F$ does not intersect the interior of $U_{(A,B)}$. We will construct a rank-$r$ matrix $M^{\prime}$ arbitrarily close to $M$ with $\operatorname{rank}_{+}(M^{\prime})>r$. For cone $P_{A}\subseteq\mathbb{R}^{r}$, let $P_{A}^{\vee}\subseteq(\mathbb{R}^{r})^{*}$ denote the dual cone, which consists of all linear functionals that are nonnegative on $P_{A}$, and similarly let $P_{B^{T}}^{\vee}$ be the dual cone of $P_{B^{T}}$. Neither the cone $P_{A}$ nor $P_{B^{T}}$ contains a line since after a change of coordinates each are a subspace intersected with a positive orthant. Therefore we can choose functionals $x$ and $y$ in the interiors of $P_{A}^{\vee}$ and $P_{B^{T}}^{\vee}$ respectively. The functional $x$ has the property that for any non-zero $v\in P_{A}$, $xv>0$, and similarly for $y$ with respect to $P_{B^{T}}$.

Let $X$ be the $m\times r$ matrix with $x$ in every row, and $Y$ the $r\times n$ matrix with $y$ in every column. Choose vectors $v$ and $w$ in the interiors of $P_{A}$ and $P_{B^{T}}$ respectively. Let $A^{\prime}=A-\epsilon X$ and $B^{\prime}=B-\epsilon Y$ for $\epsilon>0$ chosen small enough so that $v$ and $w$ are still in the interiors of $P_{A^{\prime}}$ and $P_{(B^{\prime})^{T}}$ respectively. Then $U_{(A^{\prime},B^{\prime})}$ contains the point given by $r$ copies of $v$ and $r$ copies of $w$ that is in $U_{(A,B)}$.

Let $(C,D)$ be any non-zero point on the boundary of $U_{(A,B)}$, so either $a_{i}c_{j}=0$ for some row $a_{i}$ of $A$ and column $c_{j}$ of $C$ or $d_{i}b_{j}=0$ for some row $d_{i}$ of $D$ and column $b_{j}$ of $B$. Without loss of generality assume the first case. Letting $a^{\prime}_{i}$ denote the $i$th row of $A^{\prime}$ we have $a^{\prime}_{i}c_{j}=a_{i}c_{j}-\epsilon xc_{j}<0$ because $c_{j}\in P_{A}$. This implies $(C,D)$ is outside of the cone $U_{(A^{\prime},B^{\prime})}$. Since $U_{(A^{\prime},B^{\prime})}\setminus\{0\}$ intersects the interior of $U_{(A,B)}$ but not its boundary, it must be contained in the interior of $U_{(A,B)}$. Since $F$ does not intersect the interior of $U_{(A,B)}$ or the origin, it does not intersect $U_{(A^{\prime},B^{\prime})}$. Therefore $M^{\prime}=A^{\prime}B^{\prime}$ has $\operatorname{rank}_{+}(M^{\prime})>r$.

Note that $M^{\prime}=M-\epsilon(XB+AY)+\epsilon^{2}(XY)$, which can be made arbitrarily close to $M$ in 2-norm by choosing $\epsilon$ small enough. For sufficiently small $\epsilon$, $A^{\prime}$ and $B^{\prime}$ have full rank since this is an open condition, so $\operatorname{rank}(M^{\prime})=r$. ∎

Proposition 5.4**.**

Positive $M$ has a strictly positive rank factorization if and only if the set of nonnegative rank factorizations of $M$ contains a nonempty subset that is open in the Euclidean subspace topology on $\mu^{-1}(M)$ (or equivalently the Zariski closure of $\mu^{-1}(M)\cap(\mathbb{R}_{r}^{m\times r})_{\geq 0}\times(\mathbb{R}_{r}^{r\times n})_{\geq 0}$ is $\mu^{-1}(M)$).

Proof.

First we show that the set $F$ is not contained in any facet hyperplane of $U_{(A,B)}$. Every facet $H$ of $U_{(A,B)}$ is defined by a linear equation involving either only the first set of coordinates or only the second set. Consider the former case without loss of generality. Recall that $F$ is the graph of the inverse function on $r\times r$ matrices, so the first set of coordinates are algebraically independent in $F$. Therefore $H\cap F$ has strictly lower dimension than $F$.

Suppose an open neighborhood of $F$ is contained in $U_{(A,B)}$. If the neighborhood is contained in the boundary of $U_{(A,B)}$ then $F$ is contained in the hyperplane of one of the facets since $F$ is irreducible. As shown above, this cannot happen so there must be a point on $F$ in the interior of $U_{(A,B)}$. Conversely, if $F\cap\operatorname{int}(U_{(A,B)})$ is non-empty, it is open in the subspace topology on $F$ since $\operatorname{int}(U_{(A,B)})$ is open. ∎

Suppose $\operatorname{rank}_{+}(M)=r$, and that $(A,B)$ is a nonnegative factorization. The point $(I,I)\in F$ has $\nu_{(A,B)}(I,I)=(A,B)$. To understand the possible boundary components of sets of matrices with rank and nonnegative rank equal to $r$, it is sufficient to understand the ways that $F$ and $U_{(A,B)}$ can intersect in a neighborhood of $(I,I)$. It is not true that if $F$ and $\operatorname{int}(U_{(A,B)})$ are disjoint in a neighborhood of $(I,I)$, then $M$ is on the boundary of $\mathcal{M}_{r}^{m\times n}$; they may intersect elsewhere. However, the following corollary to Lemma 4.5 demonstrates we can always construct $M^{\prime}=A^{\prime}B^{\prime}$ that has $M$ as a submatrix, is on the boundary, and for which $U_{(A^{\prime},B^{\prime})}$ agrees with $U_{(A,B)}$ in a neighborhood of $(I,I)$.

Corollary 5.5**.**

Suppose positive matrix $M$ has a nonnegative factorization $(A,B)$ such that all nonnegative factorizations of $M$ in a neighborhood of $(A,B)$ have at least one zero entry. Then there is a matrix $A^{\prime}\in\mathbb{R}_{r}^{m^{\prime}\times r}$ obtained by adding at most $r$ strictly positive rows $A$ and a matrix $B^{\prime}\in\mathbb{R}_{r}^{r\times n^{\prime}}$ obtained by adding at most $r$ strictly positive columns to $B$, such that $M^{\prime}=A^{\prime}B^{\prime}$ is on the non-trivial boundary of $\mathcal{M}_{r}^{m^{\prime}\times n^{\prime}}$.

We now consider the tangent space of $F$ at $(I,I)$, and how it intersects $U_{(A,B)}$. The tangent space of $F$ at $(I,I)$ is

[TABLE]

The cone $W_{(A,B)}$ from Section 3 is the projection to the first $\mathbb{R}^{r^{2}}$ factor of tangent directions $(D,-D)$ such that the line $(I+tD,I-tD)$ stays in $U_{(A,B)}$ for $t\in[0,\epsilon)$ for some $\epsilon>0$. The tangent directions along the diagonal matrices $D$ always lie in $W_{(A,B)}$. We recall that a nonnegative factorization $(A,B)$ is infinitesimally rigid if $W_{(A,B)}$ consists only of the diagonal matrices, and it is locally rigid if a neighborhood of $(I,I)$ in $F\cap U_{(A,B)}$ has dimension $r$, the minimal possible dimension.

Proposition 5.6**.**

If $W_{(A,B)}$ has full dimension $r^{2}$, then $M$ is in the interior of $\mathcal{M}_{r}^{m\times n}$.

Proof.

As in the proof of Proposition 5.4, if $W_{(A,B)}$ has full dimension, then the tangent space $T_{(I,I)}F$ intersects the interior of $U_{(A,B)}$ in a neighborhood of $(I,I)$. This implies that $F$ itself intersects the interior of $U_{(A,B)}$. By Propositon 5.3 and Proposition 5.4, $M$ is in the interior of $\mathcal{M}_{r}^{m\times n}$. ∎

In Example 4.11, a neighborhood of $(I,I)$ in $F\cap U_{(A,B)}$ has dimension $r$, but $\dim W_{(A,B)}>r$. In general, if $r<\dim W_{(A,B)}<r^{2}$ then this value may differ from the dimension of a neighborhood of $(I,I)$ in $F\cap U_{(A,B)}$ in either direction.

Example 5.7**.**

Consider the following rank 3 matrix with nonnegative rank 3 factorization

[TABLE]

Here $W_{(A,B)}^{\vee}$ is the conic combination of the 6 vectors

[TABLE]

corresponding to the 6 zeros in $A$ and $B$. This forms a 5 dimensional subspace of $\mathbb{R}^{9}$ and $W_{(A,B)}$ is the orthogonal complement which is a space of dimension 4 (the 3 trivial diagonal directions plus 1),

[TABLE]

However any neighborhood of $(I,I)$ in $F\cap U_{(A,B)}$ has full dimension 9. In fact $M$ is not on the algebraic boundary of $\mathcal{M}_{3}^{3\times 3}$. The geometry of the nested polytopes of this example are shown in Figure 2.

Suppose that factorization $(A,B)$ is not locally rigid, so $F\cap U_{(A,B)}$ has dimension larger than $r$ in a neighborhood of $(I,I)$. If we suppose also that $(A,B)$ is a boundary factorization, then locally $F\cap U_{(A,B)}$ cannot exceed the cone $W_{(A,B)}$, which represents the local intersection of the tangent space $T_{(I,I)}$ and $U_{(A,B)}$ (in contrast to Example 5.7). Within this situation, there are two broad cases to consider: either $F\cap U_{(A,B)}$ is equal to $W_{(A,B)}$ in a neighborhood of $(I,I)$, or it is strictly contained in $W_{(A,B)}$. We will study the first case in Section 5.2. An example of the second case is the locally rigid nonnegative factorization that is not infinitesimally rigid in Example 4.11.

5.2 Partially infinitesimally rigid factorizations

Here we present a construction to produce matrices of rank $r>3$ that are on the non-trivial boundary of nonnegative rank $r$, and have a positive dimensional set of nonnegative factorizations.

Definition 5.8**.**

A nonnegative factorization $(A,B)$ is partially infinitesimally rigid if $W_{(A,B)}$ is equal to $F\cap U_{(A,B)}$ in a neighborhood of $(I,I)$, and $\dim W_{(A,B)}<r^{2}$.

Partially infinitesimally rigid factorizations generalize infinitesimally rigid factorizations. When $\dim W_{(A,B)}$ exceeds $r$, the factorization $(A,B)$ is not rigid. In the examples we have encountered, the nonnegative factorizations in a neighborhood of $(A,B)$ have some columns of $A$ fixed, while others have freedom.

For $F$ to contain the cone $W_{(A,B)}$, it must contain its affine hull, so we first examine the question: what affine linear spaces passing through $(I,I)$ are contained in $F$? A line through $(I,I)$ has the form

[TABLE]

To be contained in $F$, it must be that $(I+tD)(I+tE)=I$. This holds exactly when $E=-D$ and $D^{2}=0$. Therefore an affine linear space in $F$ through $(I,I)$ has the form

[TABLE]

where $V$ is some linear space of $r\times r$ matrices $D$ satisfying $D^{2}=0$.

One way to produce such a space $V$ is to choose a subspace $S\subseteq\mathbb{R}^{r}$ and define

[TABLE]

However not all spaces $V$ have this form, as the following example shows. We do not know a full characterization of such spaces $V$.

Example 5.9**.**

Let $V$ be the space

[TABLE]

Each matrix $D\in V$ has $\operatorname{im}D=\ker D=\langle e_{1},te_{2}-se_{3}\rangle$, so there is no uniform space $S\subseteq\mathbb{R}^{4}$ such that $\operatorname{im}D\subseteq S\subseteq\ker D$ for all $D\in V$.

We focus on the case of a space $V_{S}$ with $S$ a coordinate subspace of $\mathbb{R}^{r}$ because we have a simple procedure to create factorizations $(A,B)$ for which $W_{(A,B)}$ has this form.

Proposition 5.10**.**

Let $(A,B)$ be an infinitesimally rigid nonnegative rank-$r$ factorization. There is a partially infinitesimally rigid nonnegative rank $r+1$ factorization $(A^{\prime},B^{\prime})$ where $A^{\prime}$ is a $n\times(r+1)$ matrix obtained from $A$ by adding a positive column and $B^{\prime}$ is a $(r+1)\times(m+1)$ matrix obtained from $B$ by adding a row of zeros and then a positive column.

Proof.

Let $S$ be $\langle e_{1},\ldots,e_{r}\rangle$. Then $V_{S}$ consists of matrices that are supported only in the first $r$ entries of the last column. We will construct the positive column added to $A$ such that

[TABLE]

This is equivalent to showing that $W_{(A^{\prime},B^{\prime})}^{\vee}$ is equal to the space of $(r+1)\times(r+1)$ matrices supported on the off-diagonal entries of the first $r$ columns.

The positive column added to $B^{\prime}$ is only to bring $B^{\prime}$ up to full rank, $r+1$. It does not contribute to $W_{(A^{\prime},B^{\prime})}^{\vee}$ and will not arise again in the proof.

First we show that the linear span of $W_{(A^{\prime},B^{\prime})}^{\vee}$ is equal this space of matrices. We characterize the generating set of $W_{(A^{\prime},B^{\prime})}^{\vee}$ coming from the zeros of $A^{\prime}$ and $B^{\prime}$. The natural embedding of each generator of $W_{(A,B)}^{\vee}$ of the form $-e_{i}b_{j}^{T}$ is a generator of $W_{(A^{\prime},B^{\prime})}^{\vee}$ since the columns of $B^{\prime}$ are the columns of $B$ with a zero entry added to the end. Each generator of the form $a_{j}^{T}e_{i}^{T}$ corresponds to $a_{j}^{T}e_{i}^{T}+a_{j,r+1}e_{r+1}e_{i}^{T}$ in $W_{(A^{\prime},B^{\prime})}^{\vee}$. In addition, $W_{(A^{\prime},B^{\prime})}^{\vee}$ has generator $-e_{r+1}b_{j}^{T}$ for each $j=1,\ldots,m$ coming from the new zero row added to $B^{\prime}$. It follows that $W_{(A^{\prime},B^{\prime})}^{\vee}$ is contained in the space claimed. The generators of the form $-e_{r+1}b_{j}^{T}$ span $V_{S}^{T}$ since $B$ has full rank $r$. Under the natural projection $\mathbb{R}^{(r+1)\times(r+1)}\to\mathbb{R}^{r\times r}$, the generating set of $W_{(A^{\prime},B^{\prime})}^{\vee}$ maps to the generating set of $W_{(A,B)}^{\vee}$ and zero, which span the $r\times r$ matrices with zero diagonal. Therefore $W_{(A^{\prime},B^{\prime})}^{\vee}$ spans the matrices supported on the off-diagonal entries of the first $r$ columns.

To prove that $W_{(A^{\prime},B^{\prime})}^{\vee}$ is a linear space, we show that zero is a strictly positive combination of the generators, and therefore zero is in the relative interior. Since $W_{(A,B)}^{\vee}$ is a linear space, zero is a positive combination of its generators,

[TABLE]

where $S_{j}$ is the set of zeros in $a_{j}$ and $T_{j}$ the set of zeros in $b_{j}$. Let $v$ denote the same positive combination of the corresponding generators of $W_{(A^{\prime},B^{\prime})}^{\vee}$,

[TABLE]

[TABLE]

The matrix $v$ is strictly positive on the first $r$ entries of the last row and zero elsewhere, and its positive entries depend on the new positive entries chosen for $A^{\prime}$. The convex cone $\operatorname{cone}(b_{1},\ldots,b_{m})\subseteq\mathbb{R}^{r}$ is full dimensional and contained in the positive orthant. Choose a vector $w$ in the interior of the cone, so it can be expressed as a strictly positive combination of the columns of $B$. We choose the entries $a_{1,r+1},\ldots,a_{n,r+1}$ so that

[TABLE]

Then $-e_{r+1}w^{T}$ is a positive combination of the generators of $W_{(A^{\prime},B^{\prime})}^{\vee}$ of the form $-e_{r+1}b_{j}^{T}$ and

[TABLE]

Thus zero is a positive combination of all the generators.

Finally, to conclude that $(A^{\prime},B^{\prime})$ is partially infinitesimally rigid, we show that $W_{(A^{\prime},B^{\prime})}$ is contained in $F$. After modding out by the diagonal scaling directions, $W_{(A^{\prime},B^{\prime})}$ is equal to $V_{S}$, so its elements square to zero. ∎

An non-trivial algebraic boundary component of $\mathcal{M}^{m\times n}_{r}$ consisting of matrices with infinitesimally rigid factorizations $(A,B)$ is defined by $r^{2}-r+1$ zero conditions on $(A,B)$. The above construction gives a recipe to produce non-trivial algebraic boundary components consisting of matrices with partially infinitesimally rigid decompositions $(A,B)$ that is also defined by zero conditions on $(A,B)$. However, the number of zero conditions is generally fewer. On the other hand, each matrix has a higher dimensional space of nonnegative factorizations.

The following example demonstrates a matrix and its partially infinitesimally rigid factorization on the algebraic boundary of $\mathcal{M}^{m\times n}_{4}$. While infinitesimally rigid factorizations for rank 4 have at least 13 zeros, this example has only 10. The space of nonnegative factorizations in its neighborhood after modding out by diagonal scaling is 3 rather than zero.

Example 5.11**.**

Let $M\in\mathcal{M}^{4\times 3}_{3}$ be the matrix with nonnegative factorization

[TABLE]

It can be checked that $W_{(A,B)}$ consists only of the diagonal, so this factorization is infinitesimally rigid. While $M$ is not on the boundary because it has only 3 columns, it could be expanded into a boundary instance by adding postive rows and columns per Lemma 4.5.

We apply the construction of Proposition 5.10 to get $M^{\prime}=A^{\prime}B^{\prime}$ with

[TABLE]

It can be checked that modulo the diagonal, $W_{(A^{\prime},B^{\prime})}$ is the space of matrices supported on entries $(1,4),(2,4),(3,4)$, so the factorization is partially infinitesimally rigid. $M^{\prime}$ is also not on the boundary of $\mathcal{M}^{4\times 4}_{4}$ but can be expanded into a boundary instance with the same zero pattern. In the space of nonnegative factorizations of $M^{\prime}$ in a neighborhood of $(A^{\prime},B^{\prime})$, the last column of $A^{\prime}$ has full dimensional freedom, while the other entries are fixed except for the diagonal action. The variation of the last column of $A^{\prime}$ varies the last column of $B^{\prime}$ while the other entries of $B^{\prime}$ are also unchanged.

The geometric picture of nested polytopes, $P^{\prime}\subseteq\Delta^{\prime}\subseteq Q^{\prime}$, for $(A^{\prime},B^{\prime})$ is shown in Figure 3. The 3-simplex $\Delta^{\prime}$ shares a facet $\Delta$ with $P^{\prime}$. Slicing along the affine span of $\Delta$ recovers the nested polygons $P\subseteq\Delta\subseteq Q$ associated to $(A,B)$. The facet $\Delta$ of $\Delta^{\prime}$ is locked in place by this lower dimensional configuration. On the other hand, the vertex of $\Delta^{\prime}$ opposite $\Delta$ is locally free to move in a 3-dimensional neighborhood of its position.

We note that in Figure 8 of [23] Mond, Smith and van Straten allude to the existence of configurations like Example 5.7, but they do not ellaborate further on their properties or construction.

Question 5.12**.**

Are there non-trivial boundary components of $\mathcal{M}^{m\times n}_{r}$ for $r\geq 4$ consisting of matrices with non-isolated partially infinitesimally rigid factorizations that do not come from the construction of Proposition 5.10 or its dual?

6 Symmetric matrices and completely positive rank

In this section we adapt our results to the case of symmetric matrices. Let $M$ be an $n\times n$ real nonnegative symmetric matrix. The completely positive rank of $M$, denoted $\operatorname{cp\text{-}rank}M$, is the smallest $r$ such that $M=AA^{T}$ for some nonnegative $n\times r$ matrix $A$ [1]. For fixed $r$ and $n$ we examine the set of symmetric matrices $M$ with rank and completely positive rank both equal to $r$, as a subset of the symmetric $n\times n$ matrices of rank $r$.

Let $M$ have rank $r$, and $A$ be a symmetric rank factor of $M$, meaning that $M=AA^{T}$ and $A$ is a $n\times r$ matrix. The set of all symmetric rank factors of $M$ is

[TABLE]

where $\operatorname{O}(r)$ is the orthogonal group on $\mathbb{R}^{r}$, which consists of the matrices $C$ that satisfy $C^{-1}=C^{T}$. Fixing $A$, we can then identify $\operatorname{O}(r)$ with the set of rank factors of $M$ by the linear map $C\mapsto AC$. Let

[TABLE]

$M$ has completely positive rank $r$ if and only if $\operatorname{O}(r)\cap U_{A}$ is not empty.

Now we suppose that $\operatorname{cp\text{-}rank}M=r$. To understand the rigidity of a nonnegative factor $A$, we study $C$ in a neighborhood of $I\in\operatorname{O}(r)$ that satisfy $AC\geq 0$. The infinitesimal motions of $A$ consist of the tangent directions such that

[TABLE]

The tangent space of $\operatorname{O}(r)$ at $I$, denoted $T_{I}\operatorname{O}(r)$, consists of all skew symmetric $r\times r$ matrices $D$, which we identify with $\mathbb{R}^{\binom{r}{2}}$ by the coordinates above the diagonal, $d_{ij}$ with $i<j$. The directions that satisfy Equations 6.1 are $AD$ where $D\in T_{I}\operatorname{O}(r)$. Let $W_{A}\subseteq T_{I}\operatorname{O}(r)$ be the cone of tangent directions such that $A(I+tD)$ is an infinitesimal motion. As in the nonsymmetric case, $W_{A}$ is cutout by linear inequalities coming from the zero entries of $A$. If row $a_{i}^{T}$ has a zero in entry $j$, it imposes condition $d_{j}^{T}a_{i}\geq 0$ where $d_{j}$ is the $j$th column of $D$. Unlike in the nonsymmetric case, a factor $A$ of $M$ has no trivial deformations, so we have the following definitions.

Definition 6.1**.**

A nonnegative factor $A$ is locally rigid if it is an isolated solution to $M=AA^{T}$ and $A\geq 0$. $A$ is infinitesimally rigid if it has no infinitesimal motions.

All of the theorems from Section 3 have analogous statements for symmetric matrices and completely positive rank. The corresponding results follow.

Proposition 6.2**.**

$A$* is infinitesimally rigid if and only if $W_{A}^{\vee}\cong\mathbb{R}^{\binom{r}{2}}$.*

Theorem 6.3**.**

If $A$ is an infinitesimally rigid nonnegative rank-$r$ factor then

• •

$A$* has at least $(r^{2}-r)/2+1$ zeros and*

• •

for every distinct pair $i,j$ taken from $1,\ldots,r$, there must be a row of $A$ with a zero in position $i$ and not in position $j$.

Proof.

If $W_{A}^{\vee}\cong\mathbb{R}^{\binom{r}{2}}$, then it must have at least $\binom{r}{2}+1$ cone generators. The generators of $W_{A}^{\vee}$ are in bijection with the zeros in $A$.

For each coordinate $d_{ij}$ with $i<j$ there must be at least one generator of $W_{A}^{\vee}$ with a strictly positive value there, and one with a strictly negative value. Since $A$ is nonnegative, to get a positive value in coordinate $d_{ij}$ requires $A$ to have a row with zero in the $j$th entry and a positive value in the $i$th entry. To get a negative value requires $A$ to have a row with zero in the $i$th entry and a positive value in the $j$th entry, since $d_{ji}=-d_{ij}$. ∎

Corollary 6.4**.**

If $A$ is an infinitesimally rigid nonnegative rank-$r$ factor with exactly $(r^{2}-r)/2+1$ zeros, then $M$ is strictly positive.

Proof.

If $A$ is infinitesimally rigid, then the dual cone $W^{\vee}_{A}\cong\mathbb{R}^{(r^{2}-r)/2}$. If $A$ has only $(r^{2}-r)/2+1$ zeros, the corresponding generating set of $W^{\vee}_{A}$ is minimal, so the only linear relation among the generators must be among all $(r^{2}-r)/2$.

If $AA^{T}$ has a zero in entry $ij$ then rows $a_{i}$ and $a_{j}$ of $A$ have zeros in complementary positions so that $a_{i}a_{j}^{T}=0$. Since the support of $a_{j}$ is contained in the set of columns for which $a_{i}$ is zero, the outer product matrix $a_{i}^{T}a_{j}$ can be expressed as a nonnegative combination of the dual vectors coming from $a_{i}$. Similarly, the matrix $-a_{i}^{T}a_{j}$ can be expressed as a nonnegative combination of the dual vectors coming from $a_{j}$. Summing these gives a linear relation among a strict subset of the generators, which is a contradiction. ∎

Corollary 6.5**.**

If $A$ is an infinitesimally rigid nonnegative factor, then there is at least one zero in every column of $A$.

Corollary 6.6**.**

If $M$ is strictly positive and $A$ is an infinitesimally rigid nonnegative rank-$r$ factor of $M$, then there are at most $r-2$ zeros in every row of $A$.

Proof.

Since $M$ is positive, no row of $A$ can contain only zeros. If a row of $A$ contains $r-1$ zeros, then there is a column of $A$ that does not contain any zero, because otherwise $AA^{T}$ would have a zero entry. This contradicts Corollary 6.5. ∎

Lemma 6.7**.**

If $A$ is an infinitesimally rigid nonnegative rank-$r$ factor with $(r^{2}-r)/2+1$ zeros, then there are at most $r-1$ zeros in every column of $A$.

Proof.

As in the proof of Corollary 6.4, the only linear relation among the generators of $W^{\vee}_{(A,B)}$ must be among all $(r^{2}-r)/2+1$ generators. If there were $r$ zeros in the same column of $A$, then there would be $r$ generators of $W^{\vee}_{A}$ contained in a $r-1$ dimensional subspace, implying a smaller linear relation which is impossible. ∎

Lemma 6.8**.**

Let $A$ be an infinitesimally rigid nonnegative rank-$r$ factorization with $(r^{2}-r)/2+1$ zeros. Let $\alpha\subseteq[r]$ and suppose $A$ has a $k\times|\alpha|$ submatrix of zeros with columns $\alpha$. Then

[TABLE]

Proof.

As in the proof of Corollary 3.6, a generating set of size $(r^{2}-r)/2+1$ is minimal, so the only linear relation among the generators must be among all of them. It can be checked that the zeros of $A$ described above correspond to $k|\alpha|$ generators of $W^{\vee}_{A}$ supported on entries $([r]\setminus\alpha)\times\alpha$. The number of generators cannot exceed the number of entries they are supported on, which gives the inequality

[TABLE]

∎

Let $Z_{A}$ be the matrix whose columns are the generators of $W_{A}^{V}$ and let $c$ be the number of columns of this matrix.

Proposition 6.9**.**

Let $M$ be a rank-$r$ matrix. If $AA^{T}$ is a size-$r$ completely positive factorization of $M$ that is locally rigid but not infinitesimally rigid, then $\text{K-}rank(Z_{A})<\min(c,\binom{r}{2})$.

The proof of this proposition is analogous to the proof of Proposition 4.8. We remark that most other conclusions of Sections 4 and 5 can also be extended to the symmetric case, but we leave this to the enterprising reader.

Proposition 6.9 together with Proposition 6.2 gives Algorithm 2 for determining infinitesimal and local rigidity of a nonnegative matrix factorization.

Appendix A Infinitesimally rigid factorizations for $5\times 5$ matrices of nonnegative rank four

In this appendix, we will present infinitesimally rigid factorizations for $5\times 5$ matrices with positive entries and of nonnegative rank four. In particular, we will show that for every zero pattern with $13$ zeros satisfying the conditions of Theorem 3.4, there exists an infinitesimally rigid nonnegative factorization realizing this zero pattern.

We consider zero patterns up to the action that permutes the rows of $A$, simultaneously permutes the columns of $A$ and the rows of $B$, permutes the rows of $B$ and transposes $AB$. As the first step, we use Macaulay2 [17] to construct an orbit representative under this action for all zero patterns with $13$ zeros satisfying the conditions of Theorem 3.4. There are 15 such orbit representatives.

Then for every zero pattern we construct random realizations by choosing non-zero entries uniformly at random between $1$ and $1000$. Finally, we use Normaliz [3] to find nonnegative factorizations that are infinitesimally rigid based on Definition 3.1. For each of the $15$ zero patterns, we are able to construct an infinitesimally rigid realization:

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

[TABLE]

We conjecture that the $15$ nonnegative factorizations above are in fact globally rigid based on the following evidence. For each of the $15$ matrices above we ran the program by Vandaele, Gillis, Glineur and Tuyttens [29] with the simulated annealing heuristic “sa” ten times. Each run consisted of at most ten attempts and the target precision was $10^{-15}$. In $13$ out of the $15$ cases at least one out of ten runs would find a nonnegative factorization of size four. Each time when a size-four nonnegative factorization was found, it was the same as the original factorization. On average $6.3$ runs were successful finding a size-four nonnegative factorization. For the third and ninth matrix none of the runs found a size-four nonnegative factorization. The algorithm was much slower for the eighth matrix than for any other matrix in the list. Although only three runs found a nonnegative factorization of target precision in this case, all other solutions looked similar to the original solution as well. This suggests that the algorithm converges slowly for this matrix. In summary, in each of the cases, either this program could not find a nonnegative factorization of target precision or it would find the nonnegative factorization that we started with. If these matrices would have other nonnegative factorizations, we find it unlikely that this would be the case.

Vandaele, Gillis, Glineur and Tuyttens discuss in [29, Section 2] that $A$ and $B$ positive are known to increase the number of factorizations of $AB$ and hence factoring $AB$ is usually easier. This suggests that matrices with small factorization spaces are the most difficult for exact nonnegative matrix factorization algorithms. Hence one application of the $15$ matrices above could be as benchmark matrices for nonnegative matrix factorization algorithms.

We also constructed orbit representatives for all zero patterns with $13$ zeros satisfying the conditions of Theorem 3.4 and Lemma 3.9 for larger matrices such that every row of $A$ contains a zero, and the number of columns of $B$ is five or every column of $B$ contains a zero. The number of such zero patterns for each matrix size is listed in Table 1.

Differently from the $5\times 5$ case, not all of these zero patterns automatically satisfy the necessary condition in Lemma 3.10 which is more difficult to check than the necessary conditions in Theorem 3.4 and Lemma 3.9. In the case of $6\times 5$ matrices, one out of $26$ zero patterns fails the necessary condition in Lemma 3.10. It is

[TABLE]

For the rest of the $25$ zero patterns, Huanhuan Chen constructs infinitesimally rigid realizations in his Master’s thesis [5]. He also shows that for larger factorizations there does not exist an infinitesimally rigid factorization realizing every pattern of $r^{2}-r+1$ zeros satisfying the conditions of Theorem 3.4 and Lemma 3.10. He gives a stronger necessary condition for an infinitesimally rigid realization to exist and conjectures that this condition is also sufficient.

Acknowledgments. We thank two anonymous referees, Huanhuan Chen, Nicolas Gillis, Ivan Izmestiev and Arnau Padrol for helpful comments and suggestions. This material is based upon work supported by the National Science Foundation under Grant No. DMS-1439786 while the authors were in residence at the Institute for Computational and Experimental Research in Mathematics in Providence, RI, during the Fall 2018 semester. Kubjas was supported by the European Union’s Horizon 2020 research and innovation programme (Marie Skłodowska-Curie grant agreement No. 748354, research carried out at LIDS, MIT and PolSys, LIP6, Sorbonne Université). We acknowledge the computational resources provided by the Aalto Science-IT project.