Generic Cospark of a Matrix Can Be Computed in Polynomial Time

TL;DR

This paper proves that the generic cospark of a matrix, based on its sparsity pattern, can be computed efficiently in polynomial time, despite the NP-hardness of the general problem.

Contribution

The paper introduces a polynomial-time algorithm to compute the generic cospark of a matrix from its sparsity pattern, linking probabilistic properties to computational efficiency.

Findings

The generic cospark equals the maximum cospark over matrices with the same sparsity pattern.

With probability one, the cospark matches the generic cospark for matrices with entries drawn from continuous distributions.

The proposed algorithm computes the generic cospark in polynomial time.

Abstract

The cospark of a matrix is the cardinality of the sparsest vector in the column space of the matrix. Computing the cospark of a matrix is well known to be an NP hard problem. Given the sparsity pattern (i.e., the locations of the non-zero entries) of a matrix, if the non-zero entries are drawn from independently distributed continuous probability distributions, we prove that the cospark of the matrix equals, with probability one, to a particular number termed the generic cospark of the matrix. The generic cospark also equals to the maximum cospark of matrices consistent with the given sparsity pattern. We prove that the generic cospark of a matrix can be computed in polynomial time, and offer an algorithm that achieves this.

Equations

$$\underset{x}{\mathrm{minimize}}$$

$$x\ne 0,$$

$$\underset{x}{\mathrm{minimize}}$$

$$A^{\perp }x=0,x\ne 0,$$

$$|X_f|=|X_{W^{\ast }}\cup B|=|\mathcal{C}\cup \mathcal{J}\cup B|=|\mathcal{C}|+|\mathcal{J}|+|B|.$$

$$|B|\ge (n-1)-sprank(A_{X_{W^{\ast }}}^S).$$

$$|\mathcal{C}|\ge sprank\left(A_{X_{W^{\ast }}}^S\right).$$

$$|X_f|$$

$$\ge sprank(A_{X_{W^{\ast }}}^S)+|\mathcal{J}|+|B|$$

$$\ge sprank(A_{X_{W^{\ast }}}^S)+|\mathcal{J}|+(n-1)-sprank(A_{X_{W^{\ast }}}^S)$$

$$=|\mathcal{J}|+(n-1)=|\mathcal{I}|+|\mathcal{J}|=|OPT|,$$

Full text

I Introduction

The cospark of a matrix $A\in\mathbb{R}^{m\times n},m>n$111We note that the results in this paper can be straightforwardly generalized to complex numbers., denoted by cospark($A$), is defined to be the cardinality of the sparsest vector in the column space of $A$ [1]. In other words, cospark($A$) is the optimum value of the following $l_{0}$-minimization problem:

[TABLE]

where $||Ax||_{0}$ is the number of nonzero elements in the vector $Ax$. It is well known that solving (1) is an NP-hard problem. Indeed, it is equivalent to computing the spark of an orthogonal complement of $A$ [1], where the spark of a matrix is defined to be the smallest number of linearly dependent columns of it [2]. Specifically, for $A$ with a full column rank, we can find an orthogonal complement $A^{\bot}$ of it, and (1) is equivalent to

[TABLE]

and the optimal value of (2) is the spark of $A^{\bot}$, denoted by spark($A^{\bot}$). Computing spark is known to be NP hard [3].

The role of $cospark(A)$ has been studied in decoding under sparse measurement errors where $A$ is the coding matrix [1]. In particular, $\frac{cospark(A)}{2}$ gives the maximum number of errors that an ideal decoder can tolerate for exact recovery. Closely related to this is the role of $spark(A^{\bot})$ in characterizing the ability to perform compressed sensing [1] [2]. Spark is also related to notions such as mutual coherence [2][4] and Restrict Isometry Property (RIP) [1] [5] which provide conditions under which sparse recovery can be performed using $l$-1 relaxation. Last but not least, in addition to its role in the sparse recovery literature, cospark (1) also plays a central role in security problems in cyber-physical systems (see [6] among others).

In this paper, we study the problem of computing the cospark of a matrix. Although it is proven that $\eqref{cosparkprob}$ is an NP hard problem, we show that the cospark that a matrix “generically” has can in fact be computed in polynomial time. Specifically, given the “sparsity pattern”, (i.e., the locations of all the non-zero entries of $A$,) $cospark(A)$ equals, with probability one, to a particular number which we termed the generic cospark of $A$, if the non-zero entries of $A$ are drawn from independent continuous probability distributions. Then, we develop an efficient algorithm that computes the generic cospark in polynomial time.

II Preliminaries

II-A Generic Rank of a Matrix

For a matrix $A\in\mathbb{R}^{m\times n}$, we define its sparsity pattern $S=\{(i,j)|A_{ij}\neq 0,1\leq i\leq m,1\leq j\leq n\}$. Given a sparsity pattern $S$, we denote $A^{S}$ to be the set of all matrices with sparsity pattern $S$ over the field $\mathbb{R}$. Since there is a one to one mapping between $S$ and $A^{S}$, we use $S$ and $A^{S}$ interchangeably to denote a sparsity pattern in the remainder of the paper.

The generic rank of a matrix with sparsity pattern $S$ is defined as follows.

Definition 1** (Generic Rank).**

Given $S$, the generic rank of $A^{S}$ is $sprank(A^{S})\triangleq\sup_{A\in A_{S}}rank(A)$.

Clearly, if $sprank(A^{S})<n$, the optimal value of (1) is zero. We will thus focus on the case $sprank(A^{S})=n$ for the remainder of the paper.

The following lemma states that the generic rank indeed “generically” equals to the rank of a matrix [7].

Lemma 1**.**

Given $S$, $rank(A)=sprank(A^{S})$ with probability one, if the non-zero entries of $A$ are drawn from independently distributed continuous probability distributions.

II-B Matching Theory Basics

We now introduce some basics from classical matching theory [8] which are necessary for us to introduce the results in the remainder of the paper.

For a bipartite graph $G(X,Y,E)$, a subset of edges $\mathcal{N}\subseteq E$ is a matching if all the edges in $\mathcal{N}$ are vertex disjoint. A max matching from $X$ onto $Y$ is a matching with the maximum cardinality. A perfect matching from $X$ onto $Y$ is a max matching where every vertex in $Y$ is incident to an edge in the matching.

Consider a (not necessarily maximum) matching $\mathcal{N}$. A vertex is called matched if it is incident to some edge in $\mathcal{N}$, and unmatched otherwise. An alternating path with respect to $\mathcal{N}$ is a path which alternates between using edges in $E\setminus\mathcal{N}$ and edges in $\mathcal{N}$, or vice versa. An augmenting path w.r.t $\mathcal{N}$ is an alternating path w.r.t. $\mathcal{N}$ which starts and ends at unmatched vertices. With an augmenting path $P$, it can be easily shown that the symmetric difference222The symmetric difference of two sets $S_{1}$ and $S_{2}$ is defined as $S_{1}\oplus S_{2}=\left(S_{1}\cup S_{2}\right)\setminus\left(S_{1}\cap S_{2}\right)$. $\mathcal{N}\oplus P$ gives a matching with size $|\mathcal{N}|+1$.

II-C Generic Rank as Max Matching

We now introduce an equivalent definition of generic rank via matching theory. A sparsity pattern $A^{S}$ can be represented as a bipartite graph as follows [7]. Let $G(X,Y,E)$ be a bipartite graph whose a) vertices $X=\{1,2,\ldots,m\}$ correspond to all the row indices of $A^{S}$, b) vertices $Y=\{1,2,\ldots,n\}$ correspond to all the column indices of $A^{S}$, and c) edges in $E=S$ correspond to all the non-zero entries of $A^{S}$. Accordingly, we also denote the bipartite graph for a sparsity pattern $S$ by $G(X,Y,S)$.

The following lemma states the equality between $sprank(A^{S})$ and the max matching of $G(X,Y,S)$ [7].

Lemma 2**.**

Given $G(X,Y,S)$, the generic rank $sprank(A^{S})$ equals to the cardinality of the maximum bipartite matching on $G$.

Accordingly, finding a max matching on this graph using the Hopcroft-Karp algorithm allows us to find the generic rank with $\mathcal{O}(|S|\sqrt{m+n})$ complexity [9].

III Generic Cospark

Similarly to the supremum definition of generic rank (cf. Definition 1), given the sparsity pattern of a matrix, we define generic cospark as follows.

Definition 2** (Generic Cospark).**

Given $S$, the generic cospark of $A^{S}$ is $spcospark(A^{S})\triangleq\sup_{A\in A_{S}}cospark(A)$.

In a spirit similar to the multiple interpretations of generic rank as in Section II, we provide a probabilistic view and a matching theory based view of generic cospark as follows.

III-A Cospark Equals to Generic Cospark With Probability One

For any $T\subset[m]$, let $A_{T}$ and $A^{S}_{T}$ represent the matrix $A$ and the set of matrices $A^{S}$ restricted to the rows $T$ respectively. A class of matrices which has cospark equal to generic cospark are those which satisfy the following property:

Lemma 3**.**

Given any sparsity pattern $S$ so that $sprank(A^{S})=n$ for $A^{S}\subset\mathbb{R}^{m\times n}$, for any $A\in A^{S}$, if $rank(A_{T})=sprank(A^{S}_{T}),\forall T\subseteq[m]$, then $cospark(A)=spcospark(A^{S})$.

Proof.

Let $x^{*}=\operatorname*{argmin}_{x\neq 0}||Ax||_{0}$, and suppose $U=\{i|a_{i}x^{*}=0\}$, where $a_{i}$ is the $i$th row of $A$. Since $A_{U}x^{*}=0$, $rank(A_{U})<n$. Now consider another matrix $C\in\mathbb{R}^{m\times n}$ with sparsity pattern $S$. Since $rank(C_{U})\leq rank(A_{U})=sprank(A^{S}_{U})<n$, $\ker(C_{U})$ is also nonempty, meaning there exists a nonzero vector $h\in\mathbb{R}^{n}$ such that $C_{U}h=0$. Because $A_{U^{c}}x^{*}$ has no zero entries, we also have $||C_{U^{c}}h||_{0}\leq||A_{U^{c}}x^{*}||_{0}=||Ax^{*}||_{0}$. This means $||Ch||_{0}=||C_{U}h||_{0}+||C_{U^{c}}h||_{0}\leq||A_{U^{c}}x^{*}||_{0}=||Ax^{*}||_{0}$. Hence, if $\hat{x}=\operatorname*{argmin}_{x\neq 0}||Cx||_{0}$, it follows $cospark(C)=||C\hat{x}||_{0}\leq||Ch||_{0}\leq||Ax^{*}||_{0}=cospark(A)$, which proves the lemma. ∎

We note that the property $rank(A_{T})=sprank(A^{S}_{T}),\forall T\subseteq[m]$ is known as the matching property of matrix $A$ [10].

Now, we have the following theorem showing that the generic cospark indeed “generically” equals to the cospark.

Theorem 1**.**

Given $S$, $cospark(A)=spcospark(A^{S})$ with probability one, if the non-zero entries of $A$ are drawn from independently distributed continuous probability distributions.

Proof.

If we have a matrix $A$ with sparsity pattern $S$ whose nonzeros are drawn from independent continuous distributions, then every submatrix of rows has rank equaling generic rank w. p. 1 (cf. Lemma 1). This immediately implies $cospark(A)=spcospark(A^{S})$ w. p. 1 by Lemma 3. ∎

III-B A Matching Theory based Definition of Generic Cospark

Let $G(X,Y,S)$ be the bipartite graph corresponding to $A^{S}\subseteq\mathbb{R}^{m\times n}$. For a subset of vertices $Z\subseteq X$, we define the induced subgraph $G(Z)$ as the bipartite graph $G(Z,N(Z),\{(i,j)|i\in Z\ ,j\in N(Z)\})$, where $N(Z)$ denotes the vertices in $Y$ adjacent to the set $Z$. $G(Z)$ is essentially a bipartite graph corresponding to the submatrix $A^{S}_{Z}$. We then have the following.

Lemma 4**.**

Given G(X, Y, S), let $OPT\subset X$ be a largest subset such that the induced subgraph $G(OPT)$ has a max matching of size $n-1$. We have that \textit{spcospark(A^{S})}=m-|OPT|.

The intuition behind this matching theory based definition of spcospark($A^{S}$) is the following. To find the sparsest vector in the image of $A$, it is equivalent to find a largest set of rows in $A$, $OPT$, which span an $n-1$ dimensional subspace. With such a subset $OPT$, we can find a vector $x^{*}$ that satisfies $A_{OPT}x^{*}=0$, and it is clear that $x^{*}\in\operatorname*{argmin}_{x\neq 0}{||Ax||_{0}}$. Furthermore, based on the equivalence between generic rank and max matching from Lemma 2, we arrive at the matching theory based definition of generic cospark in Lemma 4.

IV Efficient Algorithm for Computing Generic Cospark

In this section, we introduce an efficient algorithm that computes the generic cospark. This algorithm is based on a greedy approach motivated by Lemma 4.

Given $G(X,Y,S)$, for any size $n-1$ subset of vertices $W\subset Y$, we define $X_{W}=\{x\in X|N(x)\subseteq W\}$. In other words, $X_{W}$ is the index set of rows of $A^{S}$ with a zero entry in the remaining coordinate $v=Y\setminus W$.

We use $X_{W}$ as a basis to construct a candidate solution for $OPT$. The idea is to add a maximal subset of vertices $B\subset X_{W}^{c}$ to $X_{W}$, such that $\overline{X_{W}}=X_{W}\cup B$ has a matching of size $n-1$ onto $Y$. Specifically, we keep adding vertices $t\in X_{W}^{c}$ to $B$ as long as the submatrix corresponding to the index set $X_{W}\cup B$ has generic rank no greater than $n-1$. The following lemma shows that adding a vertex to $B$ can only increase the generic rank of $X_{W}\cup B$ by at most one.

Lemma 5**.**

Given $G(X,Y,S)$, $\forall Z\subset X$ and $u\in X\setminus Z$, $sprank(A^{S}_{Z\cup\{u\}})\leq sprank(A^{S}_{Z})+1$.

Remark 1**.**

*For a given $W$, depending on the order we visit the vertices in $X_{W}^{c}$, we could end up with different sets $B$, possibly of different sizes. However, we will prove that the optimal solution is recovered regardless. *

$\overline{X_{W}},\forall W$ are the candidate solutions for OPT, and we obtain the optimal solution by choosing the $\overline{X_{W}}$ with the largest cardinality, i.e. $X_{f}=\operatorname*{argmax}_{W\subset Y,|W|=n-1}|\overline{X_{W}}|$. The generic cospark of $A^{S}$ then equals to $m-|{X_{f}}|$.

The detailed algorithm is presented in Algorithm 1.

V Proof of Optimality of Algorithm 1

In this section, we prove that Algorithm 1 indeed solves the generic cospark. It is sufficient to prove that the set $X_{f}$ returned by the Algorithm satisfies the definition of $OPT$ in Lemma 4, i.e., $X_{f}$ is a subset of vertices of the largest size such that the induced subgraph $G(X_{f})$ has a max matching of size $n-1$. Since $G(X_{f})$ by construction has a max matching of size $n-1$, it is sufficient to prove that $X_{f}$ has the largest size, i.e., $|X_{f}|=|OPT|$.

To prove this, let us consider an optimal set $OPT\subset X$. We denote by $\mathcal{M}$ the set of $n-1$ edges of a max matching of $G(OPT)$. We denote by $W^{*}\subset Y$ the set of $n-1$ vertices in $Y$ corresponding to this max matching, and denote by $v=Y\setminus W^{*}$ the remaining vertex in $Y$. We will show that, starting with $W^{*}$, Algorithm 1 will return an $X_{f}$ such that $|X_{f}|\geq|OPT|$, and hence $|X_{f}|=|OPT|$. As the notations for this section are quite involved, an illustrative diagram is plotted in Figure 1 to help clarify the proof procedure in the following.

We first partition $OPT$ into $OPT=\mathcal{I}\cup\mathcal{J}$, $\mathcal{I}\cap\mathcal{J}=\emptyset$, where $\mathcal{I}$ is the set of $n-1$ vertices in $OPT$ corresponding to the max matching $\mathcal{M}$. Hence, $\mathcal{I}$ perfectly matches onto $W^{*}$ with $\mathcal{M}$. $\mathcal{J}$ consists of the remaining vertices in $OPT$ unmatched by $\mathcal{M}$.

WLOG, we assume $\mathcal{J}$ is nonempty. This is because, if $\mathcal{J}$ is empty, we then immediately have $|OPT|=n-1\leq|X_{f}|$.

We then have the following lemma about $\mathcal{I}$ and $\mathcal{J}$.

Lemma 6**.**

For any such partition $OPT=\mathcal{I}\cup\mathcal{J}$, we have that $\mathcal{J}\subset X_{W^{*}}$, and $\mathcal{I}\cap X_{W^{*}}$ is nonempty.

Proof.

Let $OPT$ be partitioned into $\mathcal{I}\cup\mathcal{J}$. Suppose $j\in\mathcal{J}$. If $j\notin X_{W^{*}}$, then $j$ is incident to $v$, which means $\mathcal{I}\cup\{j\}$ has a perfect matching onto $Y$. This contradicts $OPT$ has no perfect matching onto $Y$. Now suppose $\mathcal{I}\cap X_{W^{*}}$ is empty. This means every vertex in $\mathcal{I}$ is incident to $v$. Since $\mathcal{I}$ has a perfect matching onto $W^{*}$ and vertices in $\mathcal{J}$ are incident to vertices in $W^{*}$, it follows there exists an augmenting path from any vertex in $\mathcal{J}$ to $v$, which is a contradiction to $sprank(A^{S}_{OPT})=n-1$. ∎

Accordingly, we can partition $X_{W^{*}}=\mathcal{C}\cup\mathcal{J},~{}\mathcal{C}\cap\mathcal{J}=\emptyset$, with $\mathcal{C}\triangleq X_{W^{*}}\setminus\mathcal{J}$. Starting from here, the general idea of proving $|X_{f}|\geq|OPT|$ is to lower bound

[TABLE]

We immediately have the following lower bound on $|B|$:

[TABLE]

This is because a) Algorithm 1 guarantees that $sprank(A^{S}_{W^{*}\cup B})=n-1$, and b) every time we add a new vertex $t$ into $B$ (cf. Line 7 in Algorithm 1), $sprank(A^{S}_{W^{*}\cup B})$ increases by at most one (cf. Lemma 5). Since the initial generic cospark is $sprank(A^{S}_{X_{W^{*}}})$, we need at least $(n-1)-sprank(A^{S}_{X_{W^{*}}})$ vertices added into $B$ to reach $sprank(A^{S}_{W^{*}\cup B})=n-1$.

We next devote the majority of this section to provide a lower bound on $|\mathcal{C}|$.

V-A Lower Bounding $|\mathcal{C}|$

The key result we will rely on in this subsection is the following:

Theorem 2**.**

For the induced bipartite graph $G(X_{W^{*}})$, there exists a max matching that does not touch any vertices in $\mathcal{J}$.

To prove Theorem 2, we start with a partial matching $\mathcal{M}_{p}\subset\mathcal{M}$ consisting only edges that touch $\mathcal{I}\cap X_{W^{*}}$. In other words, $\mathcal{M}_{p}=\{(i,j)\in\mathcal{M}|i\in\mathcal{I}\cap X_{W^{*}}\}$. The idea is that we will build a max matching starting from $\mathcal{M}_{p}$, and this max matching will not touch any vertices in $\mathcal{J}$, thus proving Theorem 2.

We have the following two lemmas.

Lemma 7**.**

For the induced bipartite graph $G(X_{W^{*}})$ with $\mathcal{M}_{p}$ as a (not necessarily max) matching, any vertex in $N(\mathcal{J})$ is incident to some edge in $\mathcal{M}_{p}$, i.e., already matched.

Proof.

First, note any $j\in\mathcal{J}$ is not incident to $v$, so any vertex $k\in N(j)$ is in $W^{*}$. Now, for any $j\in\mathcal{J}$ and any $k\in N(j)$, we want to prove $k$ is incident to some edge in $\mathcal{M}_{p}$. Since $k\in W^{*}$, $k$ is incident to some edge $(i,k)\in\mathcal{M}$. $\mathcal{M}$ is the perfect matching from $\mathcal{I}$ to $W^{*}$, so certainly, $i$ is in $\mathcal{I}$. On the other hand, $i$ cannot be incident to $v$, or there will exist a length $3$ augmenting path from $j$ to $k$ to $i$ to $v$. Hence, $i\in X_{W}^{*}$, and the claim is proven. ∎

Lemma 8**.**

For the induced bipartite graph $G(X_{W^{*}})$ with $\mathcal{M}_{p}$ as a (not necessarily max) matching, there exists no augmenting path starting from any $j\in\mathcal{J}$.

Proof.

For any vertex $u\in N(X_{W^{*}})\setminus N(\mathcal{J})$ that is unmatched w. r. t. $\mathcal{M}_{p}$, suppose there is an augmenting path from $j$ to $u$ using edges in $\mathcal{M}_{p}$. If $u$ is unmatched in the induced graph $G(X_{W^{*}})$ w.r.t $\mathcal{M}_{p}$ and $u\in W^{*}$, then there exists an edge $(i,u)\in\mathcal{M}\setminus\mathcal{M}_{p}$ which is incident to $u$. Because $(i,u)\in\mathcal{M}\setminus\mathcal{M}_{p}$, $i$ must be incident to $v$. This means if there exists an augmenting path from $j$ to $u$ w.r.t $\mathcal{M}_{p}$, then there must exist an augmenting path from $j$ to $v$ w.r.t $\mathcal{M}$, which contradicts vertices in $\mathcal{J}$ do not have augmenting paths to $v$. ∎

Lemma 8 implies that all augmenting paths w. r. t. the partial matching $\mathcal{M}_{p}$ are from unmatched vertices in $\mathcal{C}\setminus\mathcal{I}$ (where $\mathcal{C}=X_{W^{*}}\setminus\mathcal{J}$) to unmatched vertices in $N(X_{W^{*}})\setminus N(\mathcal{J})$. A corollary which will prove useful is the following:

Corollary 1**.**

Suppose $P$ is an augmenting path from $c\in\mathcal{C}\setminus\mathcal{I}$ to $u\in N(X_{W^{*}})\setminus N(\mathcal{J})$ w. r. t. the matching $\mathcal{M}_{p}$. Then for any $j\in\mathcal{J}$, there exists no alternating path w. r. t. $\mathcal{M}_{p}$ from $j$ to any vertex in $P$.

Proof.

Let $P$ be an augmenting path from $c$ to $u$ w.r.t. $\mathcal{M}_{p}$. Suppose there exists an alternating path $P^{\prime}_{jp}$ from $j$ to a vertex $p$, where $p$ is the first vertex in $P$ encountered when traversing $P^{\prime}_{jp}$. $P^{\prime}_{jp}$ must have odd number of edges, since $p$ is a matched vertex in $P$ and $j$ is unmatched. Since $P^{\prime}_{jp}$ is odd, $p\in N(X_{W^{*}})$. Hence, if $P_{cp}\subset P$ is the restriction of $P$ from $c$ to $p$, then the alternating path $P_{cp}$ must also have odd length. The total length of $P$ must be odd since $P$ is an augmenting path, which means the length of the alternating path from $p$ to $u$ in $P$ must be even.

Since $P_{jp}$ is an odd alternating path from $j$ to $p$, and the alternating path from $p$ to $u$ in $P$ is even, then the alternating path from $P_{jp}$ to $u$ is odd. Furthermore, $j$ and $u$ are unmatched, so this path is actually an augmenting path, which immediately contradicts Lemma 8. ∎

From Corollary 1, any alternating path starting from $j$ w. r. t. $\mathcal{M}_{p}$ is vertex disjoint to any augmenting path $P$. This implies that a) any alternating path from $j$ w. r. t. $\mathcal{M}_{p}\oplus P$ remains an alternating path, and b) there remains no augmenting path starting from $j$ w. r. t. $\mathcal{M}_{p}\oplus P$, i.e., Lemma 8 continues to hold for $G(X_{W^{*}})$ with a new matching $\mathcal{M}_{p}\oplus P$.

We are now ready to prove Theorem 2.

Proof of Theorem 2.

Take $\mathcal{M}_{p}$ to be an initial matching onto $N(X_{W^{*}})$. By Lemma 7, all vertices in $N(\mathcal{J})$ are now matched, and Lemma 8 tells us we are left with augmenting paths starting from unmatched vertices in $\mathcal{C}\setminus\mathcal{I}$ to unmatched vertices in $N(X_{W^{*}})\setminus N(\mathcal{J})$. If $P_{1}$ is one such augmenting path, then $\mathcal{M}_{p}\oplus P_{1}$ is a matching with one greater cardinality. By Corollary 1, all alternating paths w.r.t $\mathcal{M}_{p}$ starting from $j$ are vertex disjoint to $P_{1}$, which implies alternating paths starting from $j$ remain unchanged. Furthermore, Corollary 1 tells us $\mathcal{M}_{p}\oplus P_{1}$ does not have augmenting paths starting from $j$. Hence, the only remaining augmenting paths are still from vertices $\mathcal{C}\setminus\mathcal{I}$ to vertices $N(X_{W^{*}})\setminus N(\mathcal{J})$. If $P_{2}$ is such an augmenting path, we can now repeat the above procedure and compute the matching $\mathcal{M}_{p}\oplus P_{1}\oplus P_{2}$. Again, alternating paths starting from $j$ remain unchanged, and $\mathcal{M}_{p}\oplus P_{1}\oplus P_{2}$ contains no augmenting paths starting from $j$. We can repeat this procedure until all augmenting paths from $\mathcal{C}\setminus\mathcal{I}$ to $N(X_{W^{*}})\setminus N(\mathcal{J})$ are eliminated. Since the final matching obtained this way has no augmenting paths, this final matching is optimal, and its edges are incident to no vertices in $\mathcal{J}$. ∎

As a result of Theorem 2, there exists a max matching of the bipartite graph $G(X_{W^{*}})$ that, on the “left hand side” of the graph, only touches vertices in $\mathcal{C}=X_{W^{*}}\setminus\mathcal{J}$. Since the size of the max matching of $G(X_{W^{*}})$ equals to $sprank\left(A^{S}_{X_{W^{*}}}\right)$ (cf. Lemma 2), we arrive at the following lower bound on $|\mathcal{C}|$:

[TABLE]

V-B Proof of the Optimality of Algorithm 1

We now show that Algorithm 1 indeed returns the generic cospark as in the following theorem.

Theorem 3**.**

For the $X_{f}$ that Algorithm 1 returns, we have that $|X_{f}|=|OPT|$.

Proof.

By the definition of $OPT$, $|X_{f}|\leq|OPT|$. To prove $|X_{f}|\geq|OPT|$, starting from (3),

[TABLE]

where (7) is from (5), and (8) is from (4). ∎

VI Algorithm Complexity

We now show that Algorithm 1 is efficient, and provide an upper bound on its computational complexity.

Theorem 4**.**

Given any $S$, Algorithm 1 computes $spcospark(A^{S})$ in $\mathcal{O}(nm(1+|S|))$ time.

Proof.

Observe in the pseudocode above, step 3 is over $n$ iterations. For each iteration, steps 4 to 9 are the most computationally expensive. Step 4 requires a $\mathcal{O}(m)$ scan of the rows of $A^{S}$, and step 5 requires us to compute a perfect matching using Hopcroft-Karp algorithm, which can be done in $\mathcal{O}(|S|\sqrt{m+n})$ time.

For the loop in steps 6 to 9, we do not need to recalculate $sprank(A^{S}_{\{X_{W}\cup B\}})$ every iteration. Given we know the max matching from the previous iteration, we only need to check if the new vertex $t$ added to $B$ has an augmenting path to an unmatched vertex in $Y$. Searching for this augmented path requires us to use breadth first search (BFS) or depth first search (DFS), which can be computed in $\mathcal{O}(|S|)$ time. Since there are $\mathcal{O}(m)$ iterations in the while loop, the total cost of steps 6 to 9 is $\mathcal{O}(m|S|)$.

Hence, for every iteration of step 3, the total cost is $\mathcal{O}(m+|S|\sqrt{m+n}+m|S|)=\mathcal{O}(m(1+|S|))$ since $n\leq m$. It follows immediately our total running time is $\mathcal{O}(nm(1+|S|))$. ∎

From Theorem 4, if $A^{S}$ is extremely sparse, the running time of Algorithm 1 is essentially quadratic.

Remark 2**.**

The algorithm’s bottleneck is in steps 6-9. For each row $t$ to add, we need to use a BFS. Since we need to add $\mathcal{O}(m)$ such vertices, the total complexity for these steps is $\mathcal{O}(m|S|)$ as in the above proof. To improve this complexity, we would like to detect multiple candidate rows to add to $B$ using a single BFS. Indeed, it can be shown further that steps 6-9 of Algorithm 1 can be improved to $\mathcal{O}(\sqrt{m}|S|)$ based on an idea similar to Hopcroft-Karp matching [9]. This will improve the total running time of Algorithm 1 to $\mathcal{O}(n\sqrt{m}|S|)$ . Details are omitted here.

VII Experimental Results for Verification

We compare the results from our algorithm of finding the generic cospark to a brute force algorithm of finding the cospark. Because the brute force algorithm has a computational complexity of $\mathcal{O}(m^{n})$, we limit the size of the test matrices to $m=20$ and $n=5$.

We run our comparison over 10 different sparsity levels spaced equally between zero and one. For each sparsity level, we generate 50 matrices, where the locations of the nonzero entries are chosen uniformly at random given the sparsity level, and the values of the non-zero entries are drawn from independent uniform distributions in $[0,1]$. For each of these 50 matrices, we compare the generic cospark given by Algorithm 1 versus that given by the brute force method. In every case, the solutions of both algorithms match. These results support the fact that our algorithm not only computes the generic cospark in polynomial time, but also obtains the actual cospark w. p. 1 if the non-zero entries are drawn from independent continuous probability distributions.

VIII Conclusion

We have shown that, although computing the cospark of a matrix is an NP hard problem, computing the generic cospark can be done in polynomial time. We have shown that, given any sparsity pattern of a matrix, the cospark is always upper bounded by the generic cospark, and is equal to the generic cospark with probability one if the nonzero entries of the matrix are drawn from independent continuous probability distributions. An efficient algorithm is developed that computes generic cospark in polynomial time.