Observability Robustness under Sensor Failures: a Computational Perspective

TL;DR

This paper investigates the computational complexity of maintaining system observability under sensor failures, proving NP-hardness of related problems and proposing polynomial-time solutions under certain practical assumptions.

Contribution

It introduces the first complexity analysis of sensor removal impacts on observability and offers efficient algorithms for specific cases with bounded eigenvalue multiplicities.

Findings

All problems are NP-hard, even in special cases.

Certain problems exhibit a submodular structure, limiting approximation guarantees.

Polynomial-time solutions are possible under practical eigenvalue multiplicity bounds.

Abstract

This paper studies the robustness of observability of a linear time-invariant system under sensor failures from a computational perspective. To be precise, the problem of determining the minimum number of sensors whose removal can destroy system observability, as well as the problem of determining the minimum number of state variables that need to be prevented from being directly measured by the existing sensors to destroy observability, is investigated. The first one is closely related to the ability of unique state reconstruction of a system under adversarial sensor attacks, and the dual of both problems are in the opposite direction of the well-studied minimal controllability problems. It is proven that all these problems are NP-hard, both for a numerical system and a structured system, even restricted to some special cases. It is also shown that the first problems both for a numerical system and a structured one share a cardinality-constrained submodular minimization structure, for which there is no known constant or logarithmic factor approximation in general. On the other hand, for the first two problems, under a reasonable assumption often met by practical systems, that the eigenvalue geometric multiplicities of the numerical systems or the matching deficiencies of the structured systems are bounded by a constant, by levering the rank-one update property of the involved rank function, it is possible to obtain the corresponding optimal solutions by traversing a subset of the feasible solutions. We show such a method has polynomial time complexity in the system dimensions and the number of sensors.

Equations

$$\begin{gathered} \dot{x}(t)=Ax(t)+Bu(t), \\ y(t)=Cx(t) \end{gathered}$$

$$\begin{array}[]{*{20}{l}}{\mathop{\min}\limits_{S\subseteq V}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}\left|S\right|}\\ {{\rm{s.t.}}\ (A,{C_{V\backslash S}}){\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\rm{unobservable}}}\end{array}$$

$$\begin{array}[]{*{20}{l}}{\mathop{\min}\limits_{S\subseteq V}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}\left|S\right|}\\ {{\rm{s.t.}}\ (\bar{A},{\bar{C}_{V\backslash S}}){\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\rm{structurally{\kern 2.0pt}unobservable}}}\end{array}$$

$$\begin{array}[]{*{20}{l}}{\mathop{\min}\limits_{S\subseteq\hat{V}}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}\left|S\right|}\\ {{\rm{s.t.}}\ (A,{C^{\hat{V}\backslash S}}){\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\rm{unobservable}}}\end{array}$$

$$\begin{array}[]{*{20}{l}}{\mathop{\min}\limits_{S\subseteq\hat{V}}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}\left|S\right|}\\ {{\rm{s.t.}}\ (\bar{A},{\bar{C}^{\hat{V}\backslash S}}){\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\kern 1.0pt}{\rm{structurally\ unobservable}}}\end{array}$$

$$H(\eta )=\left[X^{⊺},X^{\perp }+\eta 1_{n\times (n-k)}\right],$$

$$A=P\Gamma P^{-1},C=I_n.$$

$$\underset{S\subseteq V,|S|\le l}{\min }\mathrm{rank}\mathcal{O}(A,C_{V\backslash S}).$$

$${\rm{rank}}Y_{S\bigcup{\{j\}}}^{(\kappa)}=\left\{\begin{array}[]{l}{\rm{rank}}Y_{S}^{(\kappa)},{\rm if}{\kern 2.0pt}\overline{{\rm span}}(Y_{\{j\}}^{(\kappa)})\subseteq\overline{{\rm span}}(Y_{S}^{(\kappa)})\\ {\rm{rank}}Y_{S}^{(\kappa)}+1,{\rm if}{\kern 2.0pt}\overline{{\rm span}}(Y_{\{j\}}^{(\kappa)})\not\subseteq\overline{{\rm span}}(Y_{S}^{(\kappa)})\end{array}\right.$$

$$\bar{A}=\left[{\begin{array}[]{*{20}c}{0_{q\times q}}&{0_{q\times(n-q)}}\\ {{\bf\bar{1}}_{(n-q)\times q}}&{{\bf\bar{1}}_{(n-q)\times(n-q)}}\\ \end{array}}\right],$$

$$\begin{array}[]{l}{\rm{grank}}([{{\bar{A}}^{\intercal}},{{[\bar{C}^{\intercal}]}_{[V\backslash{S_{re}}]}}])={\rm{grank}}([{{\bar{A}}^{\intercal}},{{[\bar{C}^{\intercal}]}_{[{S_{ld}}]}}])\\ \leq{\rm{grank}}(\bar{A})+{\rm{grank}}({{[\bar{C}^{\intercal}]}_{[{S_{ld}}]}})<n-q+q=n.\end{array}$$

$$\hat{A}\!=\!\left[\!\!\begin{array}[]{cc}0_{(q-1)\times(q-1)}&0_{(q-1)\times(r+1-q)}\\ {\bf\bar{1}}_{(r+1-q)\times(q-1)}&{\bf\bar{1}}_{(r+1-q)\times(r+1-q)}\\ \end{array}\!\!\right],\hat{C}\!=\!\!\left[\!\!\begin{array}[]{c}[In({\cal G})]^{\intercal}\\ {\bf\bar{1}}_{(q-k-2)\times r}\\ \end{array}\!\!\right].$$

$$\underset{S\subseteq V,|S|\le |J_{re}^y|}{\min }\mathrm{grank}\mathbf{col}\{\bar{A},{\bar{C}}_{V\backslash S}\}$$

$$P_{r}\bar{M}P_{c}=\left[\begin{array}[]{cccc}\bar{M}_{11}&\bar{M}_{12}&\bar{M}_{13}&\bar{M}_{14}\\ 0&0&\bar{M}_{23}&\bar{M}_{24}\\ 0&0&0&\bar{M}_{34}\\ 0&0&0&\bar{M}_{44}\\ \end{array}\right],$$

$$P_{r}[\bar{A}^{\intercal},\bar{C}^{\intercal}]\left[{\begin{array}[]{*{20}c}{P_{c}}&{}\hfil\\ {}\hfil&I\\ \end{array}}\right]=\left[{\begin{array}[]{*{20}c}{\bar{A}_{11}}&{\bar{A}_{12}}&{\bar{A}_{13}}&{\bar{A}_{14}}&\vline&{\bar{B}_{1}}\\ {0}&{0}&{\bar{A}_{23}}&{\bar{A}_{24}}&\vline&{\bar{B}_{2}}\\ {0}&{0}&{0}&{\bar{A}_{34}}&\vline&{\bar{B}_{3}}\\ {0}&{0}&{0}&{\bar{A}_{44}}&\vline&{\bar{B}_{4}}\\ \end{array}}\right],$$

$${\rm grank}([\bar{A}^{\intercal},\bar{C}^{\intercal}])=\left|{{\mathcal{C}}(\left[{\begin{array}[]{*{20}c}{\bar{A}_{12}}&{\bar{A}_{13}}\\ 0&{\bar{A}_{23}}\\ \end{array}}\right])}\right|+{\rm grank}(\left[{\begin{array}[]{*{20}c}{\bar{A}_{34}}&{\bar{B}_{3}}\\ {\bar{A}_{44}}&{\bar{B}_{4}}\\ \end{array}}\right])$$

$$\left({\left[\begin{array}[]{cc}\bar{A}^{\intercal}_{34}&\bar{A}^{\intercal}_{44}\\ 0&0\\ \end{array}\right]},[\bar{B}^{\intercal}_{3},\bar{B}^{\intercal}_{4}]\right)$$

Full text

I INTRODUCTION

Modern control systems are often equipped with computer-based components, which could be vulnerable to cyber attacks [2]. As such, security has become a more and more important issue in control and estimation of cyber-physical systems, such as chemical processes, power grids, and transportation networks [3, 4, 5]. Typical attacks could be imposed on actuators, sensors or controllers of a control system. Among the related issues, one problem has attracted researchers’ interest, which is that, under what condition it is possible to uniquely reconstruct the states of a system by observations of system outputs in the presence of adversarial sensor attacks [6, 3, 4]. Particularly, it is found in [3, 6] that, *unique state reconstruction of a linear time invariant (LTI) system under $s\in{\mathbb{N}}$ attacks (i.e., the number of sensors that are affected by attack signals) over a sufficiently long time horizon is possible, if and only if that system remains observable after the removal of arbitrary sets of $2s$ sensors (see [3, Prop. 2] and [6, Theo. 1] for details). * Such condition is called $2s$-sparse observability in [4], and has been extended to nonlinear systems and the distributed secure state estimation scenarios [7, 5]. For a given system and $s\in\mathbb{N}$, verifying the aforementioned condition seems combinatorial. This motivates our first problem on observability robustness under sensor failures.

More precisely, we will consider the following problem: given an LTI system, determine the minimum number of sensors whose removal can make the resulting system unobservable. We call this problem the minimal sensor removal observability problem (minSRO). Here, sensor removal could also result from denial-of-service attacks on a specific set of sensors [2], which means that the attacked sensors cannot output any system information.

In addition to sensor removals, we will also briefly consider another type of sensor failures, that is, a subset of state variables are prevented from being directly measured by the existing sensors (in other words, these state variables are ‘blocked’ from the existing sensors). Such a scenario may happen, for example, when an agent in a multi-agent system (or in a cooperative localization/tracking task) moves into a region where it loses all communications with the existing sensors [8, 9]. And the dual scenario is that, in a leader-follower system, a follower loses all its direct connections with the leaders. To measure observability robustness in such scenarios, we will consider the problem of determining the minimum number of state variables that need to be blocked from the existing sensors to make the resulting system unobservable, which is referred to as the minimal state variable blocking observability problem (minSBO).

On the other hand, the dual of the minSRO, i.e., determining the minimum number of actuators (inputs) whose removal can destroy system controllability, is in the opposite direction of the well-studied minimal controllability problem (MCP), which is formulated as selecting the minimum number of inputs from a given set of inputs to ensure system controllability [10]. The MCP is proven to be NP-hard and can be approximated gracefully using simple greedy algorithms [11, 10]. An extension of the MCP is to determine the sparsest input matrices such that the system remains controllable against a desired level of actuator removals [12]. The MCP in [11, 13] could alternatively be formulated as selecting the minimum number of state variables to be directly actuated to ensure (structural) controllability. In this sense, the dual of the minSBO is also opposite to the MCP in [11, 13]. Although the MCP has been extensively explored recently [11, 10, 12, 13], little attention has been paid to its opposition direction.

In the literature, structural controllability often serves as an alternative notion for controllability in the generic sense, owing to its nice property of relying only on the zero-nonzero patterns of system matrices rather than their exact values [14, 15]. And because of this, many works have studied controllability/observability robustness under the structured system framework [16, 17, 18, 19, 20, 14]. For example, [16] discusses observability preservation under sensor removals, and [17] studies controllability preservation under simultaneous failures in both the edges and nodes. Controllability robustness is also measured by the number of additional inputs needed for controllability under edge/node removals in [14, 20]. The above works mainly focus on the classifications of edges and nodes according to effects of their failures on system controllability/observability, rather than the optimization problems. A similar problem to the minSRO under structured system framework with nonuniform actuator costs has been proven to be NP-hard in [19]. However, no efficient algorithm was provided therein. Another line to study controllability robustness is using strong structural controllability (SSC) [21], as SSC could be regarded as the ability to preserve system controllability for a structured system under arbitrary perturbations without changing the zero-nonzero patterns of the system matrices. In this line, [22] and [23] respectively develop the conventional SSC to undirected networks and to allowing entries that can take arbitrary values. SCC preservation under structural perturbations is recently studied in [24].

In this paper, we consider the minSRO and the minSBO, as well as their structure counterparts, from a computational perspective. The main contributions are as follows: 1) We prove that the minSROs are NP-hard both for a numerical system and a structured system, even restricted to some special cases. This confirms that verifying the $2s$-sparse observability condition in [3, 4] is computationally intractable; 2) Following 1), we show the minSBOs are NP-hard both for a numerical system and a structured one; 3) We reveal that the minSROs both for numerical and structured systems share a cardinality-constrained submodular minimization structure that is hard to approximate in general; 4) Because of 3), instead of finding approximate algorithms, we turn to providing scenarios where polynomial-time exact algorithms111Exact algorithms are algorithms that return exactly the optimal solutions. may exist. Under a reasonable constant bound assumption on the geometric multiplicities of system eigenvalues or matching deficiencies of system graphs, we show the minSROs can be solved in polynomial time by traversing a subset of the feasible solutions, both for numerical and structured systems. Particularly, this means, in contrast to the MCP in [11, 10], the computational intractability of the minSRO for a numerical system is essentially caused by the increase of system eigenvalue geometric multiplicities, rather than that of numbers of system states or sensors.

It is worth noting that independent of our work (see the preliminary version [1]), [25] recently shows the minSRO for a numerical system is NP-hard using a slightly different construction technique. One extension of the technique in our previous work [19] may also lead to the NP-hardness of the minSRO for a structured system. We will discuss their relations with this paper detailedly in the corresponding sections.

The rest of this paper is organized as follows. Section II provides the problem formulations, and Section III presents some preliminaries. Sections IV and V give the computational complexity and algorithms for the considered problems, respectively. Section VI studies the structured counterparts of the minSRO and minSBO, with Section VII providing some numerical results. The last section concludes this paper.

Notations: For an $m\times n$ (numerical or structured) matrix $M$, upon letting $J_{1}\subseteq\{1,...,m\}$ and $J_{2}\subseteq\{1,...,n\}$, $M_{J_{1},J_{2}}$ denotes the submatrix of $M$ whose rows are indexed by $J_{1}$ and columns are indexed by $J_{2}$, $M_{J_{1}}$ the submatrix consisting of rows indexed by $J_{1}$, and $M_{[J_{2}]}$ the submatrix consisting of columns indexed by $J_{2}$. By ${\bf col}\{X_{i}|_{i=1}^{N}\}$ (resp. ${\bf diag}\{X_{i}|_{i=1}^{N}\}$) we denote the composite (resp. diagonal) matrix with its $i$th row block (resp. diagonal block) being $X_{i}$, and $\overline{{\rm span}}(X)$ the space spanned by row vectors of $X$. The combinatorial number {\tiny{\left(\begin{array}[]{l}n\\ m\end{array}\right)}} is also denoted by ${\bf C}_{n}^{m}$ ($m\leq n$).

II Problem Formulations

Consider the following LTI system

[TABLE]

where $x(t)\in\mathbb{R}^{n}$, $u(t)\in\mathbb{R}^{m}$ and $y(t)\in\mathbb{R}^{r}$ are respectively the state, input and output vectors, $A\in\mathbb{R}^{n\times n}$, $B=[b_{1},...,b_{m}]\in\mathbb{R}^{n\times m}$ and $C={{\bf col}\{c_{i}|_{i=1}^{r}\}}\in\mathbb{R}^{r\times n}$ are respectively the state transition, input and output matrices, with $b_{i}\in{\mathbb{R}}^{n}$, $c_{i}\in{\mathbb{R}}^{1\times n}$. Without loss of generality, assume that every row of $C$ is nonzero. Throughout this paper, $V\doteq\{1,...,r\}$.

We say System (1) is $s$-robust observable, $s\in\mathbb{N}$, if under the removal of arbitrary $s$ sensors, the resulting system is still observable. To measure such observability robustness/resilience, the following minSRO is considered.

[TABLE]

Denote the cardinality of the optimal solution to Problem 1 by $r_{\min}$. Then, System (1) is $s$-robust observable for any integer $s\leq r_{\min}-1$. Moreover, according to [6, 3], it is possible to uniquely reconstruct the system states under $s^{\prime}$ sensor attacks with $s^{\prime}\leq\left\lfloor{\frac{r_{\min}-1}{2}}\right\rfloor$, where $\left\lfloor{\bullet}\right\rfloor$ denotes the floor of a scalar.

Next, we will consider the structure counterpart of Problem 1. This is motivated by the fact that the exact values of system matrices may sometimes be hard to know in practice due to modeling errors or parameter uncertainties. Instead, the sparsity patterns of system matrices are often easily accessible. Under this circumstance, one may be interested in the generic properties of a system, i.e., properties that hold almost everywhere in the corresponding parameter space [15]. Controllability and observability are such generic properties.

To be specific, a structured matrix is a matrix whose entries are either fixed zero or free parameters. Denote the set of all $n\times m$ structured matrices by $\{0,*\}^{n\times m}$, where $*$ denotes the free parameters. A structured system is a system whose system matrices are represented by structured matrices. This system is said to be structurally observable, if there exists one numerical realization of it (i.e., a pair of numerical matrices that has the same sparsity pattern as the structured matrices) that is observable. Let $\bar{A}$ and $\bar{C}$ be structured matrices consistent with the sparsity patterns of $A$ and $C$. We consider the following problem.

[TABLE]

By definition of structural observability, the optimal value of Problem 2 is an upper bound of that of Problem 1. As observability is a generic property, it is expected that the optimal values of these two problems tend to coincide with each other in randomly generated systems, which will also be demonstrated by the numerical experiments in Section VII.

We will also briefly consider another type of sensor failures, that is, a subset of state variables are blocked from the existing sensors (corresponding to that the associated nonzero columns of $C$ are transformed into zero). Let $\hat{V}\subseteq\{1,...,n\}$ be the set of indices of nonzero columns of $C$ (resp. $\bar{C}$). For $S\subseteq\hat{V}$, denote $C^{S}$ (resp. $\bar{C}^{S}$) by the matrix obtained from $C$ (resp. $\bar{C}$) by preserving its columns indexed by $S$ and zeroing out the rest. To measure such observability robustness, the minSBO and its structured counterpart can be formulated respectively as

[TABLE]

and

[TABLE]

Note that when $S=\hat{V}$, the resulting system will become (structurally) uncontrollable, which justifies that Problems 3 and 4 are always feasible. As mentioned earlier, Problems 3 and 4 can measure observability robustness in a cooperative localization/tracking system where partial individuals lose all communications with the sensors [8, 9].

III Preliminaries

This section presents some necessary preliminaries on (structural) observability and graph theories.

Lemma 1

[PBH test,[26]] System (1) is observable, if and only if for each $\lambda\in\mathbb{C}$, there exists no $x\in{\mathbb{C}}^{n}$, $x\neq 0$, such that $Ax=\lambda x$ and $Cx=0$.

Given System (1), suppose $A$ has $p\leq n$ distinct eigenvalues, and denote the $i$th one by $\lambda_{i}$. Let $k_{i}$ be the dimension of the null space of $\lambda_{i}I-A$, i.e., $k_{i}$ is the geometric multiplicity of $\lambda_{i}$. Let $X_{i}$ be an eigenbasis of $A$ associated with $\lambda_{i}$, that is, $X_{i}$ consists of $k_{i}$ vectors which are linearly independent spanning the null space of $\lambda_{i}I-A$. With these definitions, the following lemma is immediate from the PBH test.

Lemma 2** ([26])**

System (1) is observable, if and only if $CX_{i}$ is of full column rank for $i=1,...,p$.

Criteria for structural observability often have explicit graphical presentations. To this end, for System (1), let $\mathcal{X}=\{x_{1},...,x_{n}\}$, $\mathcal{Y}=\{y_{1},...,y_{m}\}$ denote the sets of state vertices and output vertices respectively. Denote the edges by $\mathcal{E}_{\mathcal{X},\mathcal{X}}=\{(x_{i},x_{j}):\bar{A}_{ji}\neq 0\}$, $\mathcal{E}_{\mathcal{X},\mathcal{Y}}=\{(x_{j},y_{i}):\bar{C}_{ij}\neq 0\}$. Let $\mathcal{D}(\bar{A},\bar{C})=(\mathcal{X}\cup\mathcal{Y},\mathcal{E}_{\mathcal{X},\mathcal{X}}\cup\mathcal{E}_{\mathcal{X},\mathcal{Y}})$ be the system digraph associated with $(\bar{A},\bar{C})$, and $\mathcal{D}(\bar{A})=(\mathcal{X},\mathcal{E}_{\mathcal{X},\mathcal{X}})$. A state vertex $x\in\mathcal{X}$ is said to be output-reachable, if there exists a path from $x$ to an output vertex $y\in\mathcal{Y}$ in $\mathcal{D}(\bar{A},\bar{C})$.

The generic rank of a structured matrix $M$, denoted by ${\rm grank}(M)$, is the maximum rank $M$ can achieve as the function of its free parameters. The bipartite graph associated with a matrix $M$ is given by ${\mathcal{B}}(M)=({\mathcal{R}}(M),{\mathcal{C}}(M),{\mathcal{E}}(M))$, where the left vertex set ${\mathcal{R}}(M)$ (resp. right vertex set ${\mathcal{C}}(M)$) corresponds to the row (resp. column) index set of $M$, and the edge set corresponds to the set of nonzero entries of $M$, i.e., ${\mathcal{E}}(M)=\{(i,j):i\in{\mathcal{R}}(M),j\in{\mathcal{C}}(M),M_{ij}\neq 0\}$. A matching of a bipartite graph is a subset of its edges among which any two do not share a common vertex. A vertex is said to be unmatched associated with a matching if it is not in edges of this matching. The maximum matching is the matching with the largest number of edges among all possible matchings. It is known that ${\rm grank}(M)$ equals the cardinality of the maximum matching of ${\mathcal{B}}(M)$ [27].

Lemma 3** ([15])**

The pair $(\bar{A},\bar{C})$ is structurally observable, if and only if (a) every state vertex is output-reachable in $\mathcal{D}(\bar{A},\bar{C})$, and (b) ${\rm grank}([\bar{A}^{\intercal},\bar{C}^{\intercal}])=n$.

We shall call (a) of Lemma 3 the output-reachability condition, and (b) the matching condition.

IV Computational Complexity of Problems 1 and 3

A natural question is whether Problem 1 is solvable in polynomial time. In this section, we will give a negative answer, following which the NP-hardness of Problem 3 is immediate.

Theorem 1

Problem 1 is NP-hard, even with dedicated sensors (i.e., each sensor measures only one state variable).

Our proof is based on the reduction from the linear degeneracy problem, defined in Definition 1. In [28], it is proven that this problem is NP-complete, and there are infinitely many integer matrices associated with which this problem is NP-complete.

Definition 1** ([28])**

The linear degeneracy problem is to determine whether a given $d\times p$ rational matrix $F=[f_{1},...,f_{p}]$ contains a degenerate submatrix of order $d$, i.e., $\det[f_{i_{1}},...,f_{i_{d}}]=0$, for some $\{i_{1},...,i_{d}\}\subseteq\{1,...,p\}$.

Proof of Theorem 1: Let $X$ be an arbitrary $k\times n$ integer matrix ($k<n$) with full row rank, and $\alpha_{1}=\max\{|X_{ij}|\}$. Let $X^{\bot}$ be the basis matrix of the null space of $X$, i.e., $X^{\bot}$ is an $n\times(n-k)$ matrix spanning the null space of $X$. $X^{\bot}$ can be constructed via the Gaussian elimination method in polynomial time ${O}(n^{3})$ [29]. Moreover, let $\alpha_{2}=\max\{|X^{\bot}_{ij}|\}$, and $\alpha_{\max}=\max\{\alpha_{1},\alpha_{2}\}$. Notice that the entries of $X^{\bot}$ are rational, and $X^{\bot}$ multiplied by any nonzero scalar is still a basis matrix of the null space of $X$. Hence, the encoding length of $\alpha_{2}$ (i.e., $\log_{2}\alpha_{2}$) can be polynomially bounded by $k$ and $n$; so is $\alpha_{\max}$. Define $H(\eta)$ as

[TABLE]

where ${\bf 1}_{n\times(n-k)}$ denotes the $n\times(n-k)$ matrix whose entries are all one. Then, clearly, $\det H(0)\neq 0$. We are to show that one can deterministically find a scalar $\eta^{*}$ satisfying $\eta^{*}>\alpha_{2}$ and $\det H(\eta^{*})\neq 0$ in polynomial time. Notice that $\det H(\eta)$ is a polynomial of $\eta$ with degree at most one as the coefficient matrix of $\eta$ in $H(\eta)$ is rank-one. Hence, $\det H(\eta)$ can be expressed as $\det H(\eta)=[\det H(1)-\det H(0)]\eta+\det H(0)$.222This can be seen as follows. Note $\det H(\eta)$ can be expressed as $\det H(\eta)=\beta_{0}+\beta_{1}\eta$ ($\beta_{0},\beta_{1}\in{\mathbb{C}}$). Upon solving the system of equations $\det H(0)=\beta_{0}$ and $\det H(1)=\beta_{0}+\beta_{1}$, we get $\beta_{0}=\det H(0)$ and $\beta_{1}=\det H(1)-\det H(0)$, which results in the proposed expression. Note also $\det H(\eta)$ has at most one root for $\eta$ as $\det H(0)\neq 0$. Therefore, in arbitrary set consisting of $2$ distinct rational numbers which are bigger than $\alpha_{2}$, there must exist one $\eta^{*}$, such that $\det H(0)(1-\eta^{*})+\det H(1)\eta^{*}\neq 0,$ leading to $\det H(\eta^{*})\neq 0$. Moreover, dy the definition of determinant, it holds $\det H(0)\leq\alpha_{\max}^{n}n^{n}$ and $\det H(1)\leq(\alpha_{\max}+1)^{n}n^{n}$, which means $\det H(0)$ and $\det H(1)$ have encoding lengths (being $n\log_{2}\alpha_{\max}+n\log_{2}n$ and $n\log_{2}(\alpha_{\max}+1)+n\log_{2}n$ respectively) polynomially bounded by $n$. Afterwards, let matrices $P=H(\eta^{*})$, $\Gamma={\bf{diag}}\{I_{k},2,3,...,n-k+1\}$, and construct the system as

[TABLE]

Since the encoding lengths of entries of $P$ are polynomially bounded by $n$, its inversion $P^{-1}$ can be computed in polynomial time and has polynomially bounded encoding lengths too. Hence, $A$ can be computed in polynomial time.

We claim that the optimal value of Problem 1 associated with $(A,C)$ is no more than $n-k$, if and only if there exists an $k\times k$ submatrix of $X$ which has zero determinant.

Indeed, from the construction of $A$, the $(k+i)$th column of $P$, denoted by $P_{[k+i]}$, is the eigenvector of $A$ associated with the eigenvalue $i+1$, $i=1,...,n-k$. Notice that, all entries of $P_{[k+i]}$ are nonzero as $\eta^{*}>\alpha_{2}$. Hence, all $n$ rows of $C$ need to be removed, such that the resulting $C_{V\backslash S}P_{[k+i]}$ fails to be of full column rank, where $S\subseteq V$. From Lemma 2, the optimal value of Problem 1 associated with $(A,C)$ then equals the minimum number of rows (resp., columns) whose removal from $X^{\intercal}$ (resp., $X$) makes the resulting matrix fail to have full column (row) rank. If such value is no more than $n-k$, there must exist an $k\times k^{+}$ submatrix of $X$ which is not of full row rank for some $k^{+}\geq k$. Then, the linear degeneracy problem associated with $X$ is yes.

Conversely, suppose there is an $k\times k$ submatrix of $X$ with zero determinant, and denote it by $X_{[\bar{S}]}$, $\bar{S}\subseteq V$. Then, one just needs to remove the columns indexed by $V\backslash{\bar{S}}$ from $X$, such that the resulting $X_{[\bar{S}]}$ fails to be of full row rank. Hence, the optimal value of Problem 1 is no more than $|V\backslash{\bar{S}}|=n-k$. The fact that the linear degeneracy problem associated with $X$ is NP-complete yields the NP-hardness of Problem 1. $\hfill\blacksquare$

The above result indicates that, there is in general no efficient way to verify whether a given system is $s$-robust observable for a given $s\in{\mathbb{N}}$ unless $P=NP$, even when $C=I_{n}$. This confirms the conjecture implicitly made in [6, 3, 4] where they tend to think that computing the maximal number of sensor attacks that can be tolerated is computationally intractable (see Section I). It is worth noting that an independent work [25] from the present paper (see [1, Coro. 3]) also shows it is NP-hard to verify the $s$-robust observability condition, which uses a similar technique but without the dedicated output configuration constraint. The dedicated output/input configuration might be common both in theoretical studies and practical implementations [10, 12, 11]. Therefore, it would be desirable to make clear the complexity of Problem 1 even in this case. Furthermore, with $C=I_{n}$, removing one sensor corresponds to that exactly one state variable loses its direct measurements. Hence, Theorem 1 readily leads to the following corollary.

Corollary 1

Problem 3 is NP-hard.

Next, by the duality between controllability and observability, it is easy to obtain the following corollary on controllability robustness under actuator failures/removals.

Corollary 2

For System (1), it is NP-hard to determine the minimum number of actuators whose removal can make the resulting system uncontrollable.

Taking the input and output into consideration together, we have the following conclusion.

Corollary 3

For System (1), it is NP-hard to determine the minimal total number of sensors and actuators whose removal makes the resulting system neither controllable nor observable.

Proof: Construct $(A,C)$ as suggested in the proof of Theorem 1, i.e., $A=P\Gamma P^{-1}$, $C=I_{n}$. Construct $B$ such that $B\in\mathbb{R}^{n\times k}$ and $(A,B)$ is controllable. Since the eigenbases $P$ of $A$ are available, the matrix $B$ can be constructed in polynomial time as suggested in [30]. Notice that $k$ is the maximum geometric multiplicity of eigenvalues of $A$. According to [30], the minimal number of inputs (actuators) that ensures controllability of the associated system equals $k$. Hence, removing any one of these $k$ actuators can make the system uncontrollable. As a result, the minimal total number of actuators and sensors whose removal causes uncontrollability and unobservability equals $r_{\min}+1$, where $r_{\min}$ is the optimum of the minSRO with $(A,C)$. The latter problem is shown to be NP-hard in Theorem 1. The required result follows. $\hfill\blacksquare$

V Structure and Algorithm for Problem 1

In this section, we first show that Problem 1 has a cardinality-constrained submodular minimization (CCSM, i.e., minimizing a submodular function subject to an upper/lower bound on cardinality) structure, which indicates it might be hard to approximate. We then show Problem 1 can be solved in polynomial time under a constant bound assumption on the system eigenvalue geometric multiplicities.

We briefly introduce the notion of submodularity here; see [27] for more details. Let $V$ denote a finite set. A function $f:2^{V}\rightarrow{\mathbb{R}}$ that takes a subset of $V$ as the input and outputs a real value is called submodular, if for any $S,T\subseteq V$, $f(S)+f(T)\geq f(S\cap T)+f(S\cup T)$.

V-A The CCSM Structure of Problem 1

Let ${\cal O}(A,C)$ be the observability matrix of $(A,C)$, i.e., ${\cal O}(A,C)={\bf col}\{C,CA,...,CA^{n-1}\}$. Then, it can be seen that Problem 1 can be solved by invoking the following problem for at most $O(\log_{2}r)$ times using bisection on $l\in\{0,...,r\}$ until the corresponding objective is less than $n$

[TABLE]

Define $f(S):2^{V}\rightarrow\mathbb{N}$ as $f(S)={\rm rank}{\kern 1.0pt}{\mathcal{C}}(A,C_{S})$. It is known that $f(S)$ is submodular on $S\subseteq V$ [10, Theo. 7]. From the property of submodularity [27], $f(V\setminus S)$ is also submodular on S$\subseteq V$. This means that (1) can be formalized as a CCSM problem. However, the general CCSM is hard to approximate, and there is no known constant or logarithmic factor for it. Specifically, [31] has shown that approximating the submodular function minimization over an upper bound on cardinality within a factor of $O(\sqrt{\frac{r}{{\rm ln}r}})$ is generally impossible, where $r\doteq|V|$. 333We can also reformulate Problem 1 as minimizing $|S|$ subject to ${\rm rank}{\kern 1.0pt}{\mathcal{C}}(A,C_{V\backslash S})\leq n-1$. From the bicriteria results in [31], approximating the cardinality minimization subject to a monotone submodular function value upper bound better than $O(\sqrt{\frac{lnr}{r}})$ is generally impossible, that is, upon letting $r_{\rm{alg}}$, $r_{\min}$ and $r$ be respectively the returned value by a polynomial algorithm, the optimal value, and the cardinality of $V$, making $r-r_{\rm{alg}}$ closer to $r-r_{\min}$ than a factor of $O(\sqrt{\frac{lnr}{r}})$ is impossible. Hence, in the following, we turn to providing a scenario where the polynomial-time (in $n$ and $r$) exact algorithm exists for Problem 1.

V-B Algorithm with Bounded Geometric Multiplicities

We shall assume that a collection of eigenbases of $A$ are computationally available. The symbols $p$, $X_{i}|_{i=1}^{p}$ and $k_{i}|_{i=1}^{p}$ are defined in Section III. Define $Y^{(i)}\doteq CX_{i}$, $i=1,...,p$.

We further assume that the geometric multiplicities of eigenvalues of $A$ are bounded by some fixed constant $\bar{k}\in\mathbb{N}$. That is, $k_{i}\leq\bar{k}$, $i=1,...,p$, as $n$ and $r$ increase. For most practical engineering systems, this assumption is reasonable. In fact, it is found in [32] that, with probability tending to $1$, the symmetric matrices with random entries have no repeated eigenvalues, and Erdos-Renyi (ER) random graph has simple spectrum asymptotically almost surely. Besides, in the areas of multi-agent systems or distributed estimation, Laplacian matrices of undirected graphs with bounded444Hereafter, if not specified, by being bounded we mean being bounded by a (known) constant as the input size grows. connected components, or of weakly-connected directed leader-follower networks with bounded number of leaders, tend to have the maximum geometric multiplicities of their eigenvalues that are more than one but bounded by the corresponding constants [33]. Similar scenarios may arise in modeling composite/layer networks in which the system matrices are often (Cartesian) products of smaller factor graphs [34].

Let us focus on an individual eigenvalue $\lambda_{i}$, $i\in\{1,...,p\}$. Let $r_{i}$ be the minimum number of rows whose removal from $Y^{(i)}$ makes the remaining matrix fail to be full of column rank. To determine $r_{i}$, an exhaustive combinatorial search needs to compute the ranks of at most ${\bf C}_{r}^{1}+\cdots+{\bf C}_{r}^{r-k_{i}}\to O(2^{r})$ submatrices, which increases exponentially with $r$ even when $k_{i}$ is bounded. Hence, the direct combinatorial search is not computationally efficient.

In what follows, a polynomial time algorithm based on traversals over a recursive tree is presented. Here, ‘recursive’ comes from the fact that this tree is constructed recursively. The pseudo code of this algorithm is given in Algorithm 1. The intuition behind Algorithm 1 is that, for the $\kappa$th eigenvalue of $A$, $\kappa\in\{1,...,p\}$, instead of directly determining $r_{\kappa}$, we try to determine $r-r_{\kappa}$, which is the maximum number of rows of $Y^{(\kappa)}$ that fail to have full column rank. However, since $r-r_{\kappa}$ could be as large as $r-1$, a naive combinatorial search needs to compute the ranks of at most ${\bf C}_{r}^{k_{\kappa}-1}+\cdots+{\bf C}_{r}^{r-1}$ submatrices, which again increases exponentially with $r$ even when $k_{\kappa}$ is bounded. The key idea to reduce the exponentially increasing complexity to the polynomially increasing one is leveraging the rank-one update rule expressed by (2). Algorithm 1 first builds a recursive tree and then searches the maximum return value among the leaves of this tree. An illustration of such a recursive tree is given in Fig. 1. To build the recursive tree for the $\kappa$th eigenvalue, the root is $\bar{T}^{(0)}_{1}=\emptyset$. In the $\tau$th layer555In this paper, the root of a tree is indexed as the [math]th layer., $\tau=0,...,k_{\kappa}-1$, the set $T_{i}^{(\tau)}\subseteq V$ has the property that $Y^{(\kappa)}_{T_{i}^{(\tau)}}$ has rank $\tau$, $i=1,...,L_{\tau-1}$, where $L_{\tau-1}$ is the number of nodes in the $\tau$th layer. Then, $Y^{(\kappa)}_{\bar{T}_{i}^{(\tau)}}$ is obtained by adding the maximal rows of $Y^{(\kappa)}_{V\setminus T_{i}^{(\tau)}}$ to $Y^{(\kappa)}_{T_{i}^{(\tau)}}$ while its rank is preserved. The collection of sets $\{\bar{T}_{i}^{(\tau)}|_{i=1}^{L_{\tau-1}}\}$ form the $\tau$th layer of the tree. Next, each set ${\bar{T}_{i}^{(\tau)}}$ generates $|\Omega_{i}^{(\tau)}|$ sets, constituting $\{T_{j}^{(\tau+1)}|_{j=1}^{L_{\tau}}\}$ for the $(\tau+1)$th layer, where $\Omega_{i}^{(\tau)}$ is such set whose arbitrary element $j$ makes matrix $Y^{(\kappa)}_{\bar{T}_{i}^{(\tau)}\cup\{j\}}$ have rank $\tau+1$. After $\tau=k_{\kappa}-1$, the complete recursive tree is obtained, and one just needs to search the set with maximum cardinality among the leaves of this tree, i.e., $T_{\max}^{[\kappa]}$, whose cardinality equals exactly $r-r_{\kappa}$.

Theorem 2

Given a system $(A,C)$ with geometric multiplicities of eigenvalues of $A$ bounded by $\bar{k}$, $A\in\mathbb{R}^{n\times n}$, $C\in\mathbb{R}^{r\times n}$, Algorithm 1 can determine the optimal solution to Problem 1 in time complexity at most $O({\bar{k}^{2}}{r^{\bar{k}+1}}n+r{n^{3}})$.

Proof: As analyzed above, the main procedure of Algorithm 1 is to determine the maximum number of rows of $Y^{(\kappa)}$ that fail to have full column rank for each eigenvalue $\lambda_{\kappa}$, $\kappa=1,...,p$. To justify the recursive procedure, observe that for any $S\subseteq V$, $j\in V$,

[TABLE]

Hence, for the $\kappa$th eigenvalue, the recursive tree has depth exactly $k_{\kappa}$. Moreover, $\{\bar{T}_{1}^{({k_{\kappa}}-1)},\cdots,\bar{T}_{L_{{k_{\kappa}}-2}}^{({k_{\kappa}}-1)}\}$ index all the submatrices formed by rows of $Y^{(\kappa)}$ with the property that: it has rank $k_{\kappa}-1$, and adding any rest rows from $Y^{(\kappa)}$ can make it have full column rank. The optimality of the solution returned from Algorithm 1 then follows immediately.

In the recursive tree for the $\kappa$th eigenvalue, each parent node has at most $r$ child nodes. Thus, the $\tau$th layer has at most $r^{\tau}$ nodes, $\tau=0,...,k_{\kappa}-1$. Hence, the total nodes in that tree is at most $\sum\nolimits_{\tau=0}^{{k_{\kappa}}-1}{{r^{\tau}}}=(r^{k_{\kappa}}-1)/(r-1)\to O({r^{{k_{\kappa}}-1}})<{r^{{k_{\kappa}}}}$ ($r\geq 3$). For each node, the rank update procedure incurs $O(k_{\kappa}^{2}r\cdot r)$ using the singular value decomposition (Lines 6-7 of Algorithm 1). To obtain $Y^{(\kappa)}|_{\kappa=1}^{p}$, it incurs computational complexity $O(rn^{3})$. Note that $k_{\kappa}\leq\bar{k}$ for $\kappa=1,...,p$, and $p\leq n$. Hence, the total computational complexity of Algorithm 1 is at most $O({\bar{k}^{2}}{r^{\bar{k}+1}}n+r{n^{3}})$. $\hfill\blacksquare$

Remark 1

The above results show that the computational intractability of Problem 1 is essentially caused by the increase of eigenvalue geometric multiplicities of $A$, rather than that of dimensions of states or number of sensors. By contrast, from [11, 10], it is known that the MCPs discussed therein are NP-hard even when the involved systems have no repeated eigenvalues. To accelerate Algorithm 1, one could remove the repeated elements from $\{\bar{T}_{i}^{\tau}|_{i=1}^{L_{\tau-1}}\}$ after Line 6, and collect all the rows $J\subseteq\Omega_{i}^{(\tau)}$ that make $Y^{(\kappa)}_{\bar{T}_{i}^{(\tau)}\cup J}$ have rank $\tau+1$ as a single node for each $T_{i}^{(\tau)}$ in Lines 7-8, to reduce the number of repeated nodes in the $\tau$th layer of the recursive tree.

Remark 2

It is remarkable that Algorithm 1 essentially finds the optimal solutions by traversing a subset of feasible solutions $\{V\backslash\bar{T}_{i}^{(k_{\kappa}-1)}|_{i=1,...,L_{k_{\kappa}-2}}^{\kappa=1,...,p}\}$ to the original problem. Since it is an exact algorithm, without any restriction on the problem input, it cannot be guaranteed Algorithm 1 will perform significantly better than the exhaustive combinatorial search in the worst case. A similar computational complexity to Algorithm 1 can be obtained by an alternative two-stage procedure, which consists of, first enumerating all $k_{\kappa}-1$ independent rows of $Y^{(\kappa)}$ ($\kappa=1,...,p$), and then for each element, adding the maximal number of rows from $Y^{(\kappa)}$ to it without increasing its rank of $k_{\kappa}-1$. To ensure a polynomial time complexity with bounded $k_{\kappa}$, instead of enumerating over the $r-(k_{\kappa}-1)$ rows in the second stage, one should leverage the rank-one update property (2) and adopt a way similar to Line 6 of Algorithm 1. The only difference between those two procedures is that Algorithm 1 uses a recursive (rank-one update) way to get a subset of feasible solutions. In contrast, the latter procedure directly skips to finding all rank-$(k_{\kappa}-1)$ bases. Note throughout Algorithm 1, the only computation involving the rank is to check whether a matrix can preserve its rank after adding a row (Line 6). Hence, some fast dynamic rank-one update algorithms [35, 36] can be used to reduce the rank involved computations in each iteration. This may sometimes make Algorithm 1 faster than the two-stage procedure. To be specific, directly implementing the two-stage procedure for Problem 1 incurs $O(\bar{k}^{2}r^{\bar{k}+1}n+rn^{3})$ (the best bound is $O(\bar{k}^{w-1}r^{\bar{k}+1}n+rn^{3})$), while Algorithm 1 using the dynamic rank-one update algorithm in [36] incurs $\tilde{O}(\bar{k}r^{\bar{k}+1}n+\bar{k}^{w}+rn^{3})$, where $w=2.373$ is the matrix multiplication exponent.666$\tilde{O}(f(n))$ stands for $O(f(n)(logn)^{k})$ for some positive constant $k$. Moreover, the recursive rank-one update way may be more preferable in the scenario where partial sensors are forbidden to be removed, or where the eigenvalues $\lambda_{i}|_{i=1}^{p}$ are more easily accessible than the eigenbases $X_{i}|_{i=1}^{p}$ [29] (in which case one can alternatively minimize $|S|$ to make ${\bf col}\{A-\lambda_{i}I,C_{V\backslash S}\}$ column rank deficient, by Lemma 1). In these scenarios, most rows of $C$ could be collected to $\bar{T}^{({\tau})}_{i}$ in the early iterations of Algorithm 1, leading to a relatively small $\Omega_{i}^{({\tau})}$, thus a recursive tree with relatively small number of nodes.

VI Computational Perspective of Problems 2 and 4

Algorithm 1 requires accurate eigenspace decomposition of the system matrices, which may encounter numerical instabilities or rounding errors for large-scale systems. Hence, studying the corresponding structured systems may be more practical. In this section, we first prove that Problems 2 and 4 are NP-hard, then provide the exact algorithm for Problem 2 under a corresponding assumption, and finally give a decomposition-based acceleration technique for this algorithm.

VI-A Complexity of Problems 2 and 4

Theorem 3

Problem 2 is NP-hard, even when $n\!-\!r$, i.e., the difference between the numbers of states and sensors, is arbitrarily large.

We first give some definitions and lemmas needed to proceed with our proof. Let ${\bf\bar{1}}_{m\times n}$ be an $m\times n$ structured matrix whose entries are all free parameters. A set of columns of a structured matrix are said to be linearly dependent, if the generic rank of the matrix consisting of them is less than the number of their columns. A circuit of an $m\times n$ matrix $M$, is a set $J\subseteq\{1,...,n\}$ such that ${\rm rank}(M_{[J]})=|J|-1$. The set of indices of any linearly dependent columns of $M$ contains at least one circuit of $M$. A clique of an undirected graph is a subgraph such that any two of its vertices are adjacent. The $k$-clique problem is to determine whether there is a clique with $k$ vertices, which is NP-complete [29].

Lemma 4

Let matrix $\bar{M}\in\{0,*\}^{n\times l}$. If ${\rm grank}([{\bf\bar{1}}_{n\times m},\bar{M}])<n$, where $m<n$, then ${\rm grank}(\bar{M})<n-m$.

Proof: Suppose ${\rm grank}(\bar{M})=p\geq n-m$. Then, there exist two sets $J_{1}\subseteq\{1,...,n\}$ and $J_{2}\subseteq\{1,...,l\}$ with $|J_{1}|=|J_{2}|=p$, such that ${\rm grank}(\bar{M}_{J_{1},J_{2}})\geq n-m$. Therefore, {\rm grank}([{\bf\bar{1}}_{n\times m},\bar{M}])\geq{\rm grank}(\left[{\begin{array}[]{*{20}c}{{\bf\bar{1}}_{n-p,m}}&0\\ {{\bf\bar{1}}_{p,m}}&{\bar{M}_{J_{1},J_{2}}}\\ \end{array}}\right])=\min\{n-p,m\}+p=n-p+p=n, causing a contradiction. $\hfill\blacksquare$

Proof of Theorem 3: Our proof is a reduction from the NP-complete $k$-clique problem to one instance of Problem 2. We divide it into three steps.

Step 1). Let ${\cal G}=({\cal V},{\cal E})$ be an undirected graph (without self-loop), and $|{\cal V}|=p$, $|{\cal E}|=r$. The incidence matrix of ${\cal G}$, denoted by $In({\cal G})$, is an $r\times p$ matrix so that $[In({\cal G})]_{ij}=1$ if $v_{j}\in V(e_{i})$ and [math] otherwise, where $v_{j}\in{\cal V}$, $e_{i}\in\cal E$. Construct the matrix \tilde{C}\doteq\left[In({\cal G}),{{{\bf\bar{1}}_{r\times({\tiny{\left({\begin{array}[]{*{20}{c}}k\\ 2\end{array}}\right)}}-k-1)}}}\right], and let the output matrix $\bar{C}$ be $\bar{C}=[{0_{r\times l}},\tilde{C}]$, where $k\geq 5$ (so that {\tiny{\left({\begin{array}[]{*{20}{c}}k\\ 2\end{array}}\right)}}-k-1\geq 2), ‘$1$’ in $In(\cal G)$ represents a free entry, and the integer $l\geq 0$ can be arbitrarily selected. Let q\doteq\tiny{\left({\begin{array}[]{*{20}{c}}k\\ 2\end{array}}\right)}. Without any losing of generality, assume that $p>k+1$. It demonstrates in Page 53 of [37] that: 1) Every circuit $J$ of $\tilde{C}^{\intercal}$ must satisfy $|J|\geq q$. 2) If $\cal G$ has a $k$-clique, then $\tilde{C}^{\intercal}$ has a circuit with cardinality $q$. 3) If $\tilde{C}^{\intercal}$ has a circuit with cardinality $q$, then $\cal G$ has an $k$-clique.

From the aforementioned results, it can be declared that, $\tilde{C}^{\intercal}$ (as well as $\bar{C}^{\intercal}$) has $q$ linearly dependent columns, if and only if the digraph $\cal G$ has an $k$-clique. Indeed, from Fact 1), if $\tilde{C}^{\intercal}$ has $q$ linearly dependent columns, then such columns must form a circuit of $\tilde{C}^{\intercal}$. Otherwise, $\tilde{C}^{\intercal}$ has a circuit with cardinality less than $q$. With Fact 3), the ‘only if’ direction is obtained. The ‘if’ direction comes directly from Fact 2).

Step 2). Construct the state transition matrix $\bar{A}$ as

[TABLE]

where $n\doteq l+p+q-k-1>q$, and $l$ could be selected so that $n-r$ can be arbitrarily large. Construct the system digraph ${\cal D}(\bar{A},\bar{C})$, and let state vertices ${\cal X}=\{x_{1},...,x_{n}\}$, output vertices ${\cal Y}=\{y_{1},...,y_{r}\}$. From the construction, it is clear that every state vertex in ${\cal X}\backslash\{x_{n}\}$ is an in-neighbor of $x_{n}$, and $x_{n}$ is reachable to every output in $\cal Y$. Hence, all the $r$ outputs must be deleted so that at least one state vertex is output-unreachable. In other words, the minimum number of outputs (sensors) needed to be deleted to destroy the output-reachability of at least one state vertex equals $r$ (Lemma 3).

Step 3). We declare that the optimal value of Problem 2 associated with $(\bar{A},\bar{C})$ is no more than $r-q$, if and only if there exist $q$ linearly dependent columns in $\bar{C}^{\intercal}$. Denote the optimal solution to Problem 2 by $S_{opt}\subseteq V$. For the one direction, suppose there are $q$ linearly dependent columns in $\bar{C}^{\intercal}$, and denote the set of its indices by $S_{ld}$. Let $S_{re}=V\backslash S_{ld}$. It holds that

[TABLE]

As a result, $S_{re}$ is a feasible solution to Problem 2 associated with $(\bar{A},\bar{C})$. Since $|S_{re}|=r-q$, we have $|S_{opt}|\leq r-q$.

For the other direction, suppose that $|S_{opt}|\leq r-q$. Then, since the minimum number of sensors whose removal destroys the output-reachability condition equals $r$, it must hold that ${\rm grank}([\bar{A}^{\intercal},{[\bar{C}^{\intercal}]}_{[V\backslash S_{opt}]}])<n$. As ${\rm grank}(\bar{A})=n-q$, from Lemma 4, it is valid that ${\rm grank}({[\bar{C}^{\intercal}]}_{[V\backslash S_{opt}]})<q$. Noting that $|V\backslash S_{opt}|\geq q$, we have that the columns of $\bar{C}^{\intercal}$ indexed by $V\backslash S_{opt}$ are linearly dependent.

Since we have shown that verifying whether there are $q$ linearly dependent columns in $\bar{C}^{\intercal}$ (so in $\tilde{C}^{\intercal}$) is equivalent to verifying whether the digraph $\cal G$ has an $k$-clique, which is NP-complete, and the above-mentioned reduction is in polynomial time, it is concluded that Problem 2 is NP-hard. $\hfill\blacksquare$

The key challenge to prove Theorem 3 lies in constructing a system in which there is an explicit relationship in size between the number of sensors whose removal destroys the output-reachability condition and that whose removal destroys the matching condition.

In our previous work [19], we have proved the NP-hardness of Problem 2 in the case with nonuniform sensor cost (by duality between controllability and observability), and indicated that it might be possible to extend the proof technique therein to show the computational complexity of Problem 2 with uniform cost (since the extension may not be so straightforward and much of the details is not given, the NP-hardness of Problem 2 has not been definitively declared in [19]). In that extension, by replicating one sensor $n-1$ times, where $n$ is the number of states of the constructed system instance, the number of sensors is $2n-1$. Here, we have provided a different technique yet a more practical system instance to show the NP-hardness of Problem 2, in which any two sensors have different structure configurations (i.e., they measure different combinations of state variables), and the number of state variables $n$ can be arbitrarily larger than that of the sensors $r$. Furthermore, our proof technique is more general in the sense that it can be slightly modified to show the intractability of Problem 4.

Theorem 4

Problem 4 is NP-hard.

Proof: For space consideration, we give the sketch of our proof, which follows similar reasoning to the proof of Theorem 3 with slight modifications. Given an undirected graph ${\cal G}=({\cal V},{\cal E})$, let $In({\cal G})$ (the incidence matrix), $r\ (\doteq|{\cal E}|)$, $p\ (\doteq|{\cal V}|)$, $k\ (\geq 5)$, q\ (\doteq{\tiny{\left({\begin{array}[]{*{20}{c}}k\\ 2\end{array}}\right)}}) and $\tilde{C}$ be defined in the same way as in the proof of Theorem 3. Construct a structured system $(\hat{A},\hat{C})$ as

[TABLE]

Let $S_{opt}\subseteq\{1,...,r\}$ be the set of state variables with the minimum cardinality that need to be blocked from the existing sensors in system $(\hat{A},\hat{C})$, so that the resulting system becomes structurally uncontrollable. We will show that $|S_{opt}|\leq r-q$, if and only if $\cal G$ has a $k$-clique.

For the one direction, suppose that $\cal G$ has a $k$-clique. Then, from Step 1) in the Proof of Theorem 3, $\tilde{C}$ has $q$ linearly dependent rows, whose row index set is denoted by $S_{ld}$ ($|S_{ld}|=q$). Using Lemma 4 on $\tilde{C}_{S_{ld}}$, we get ${\rm grank}(\hat{C}_{[S_{ld}]})<q-1$, noting that ${\rm grank}(\tilde{C}_{S_{ld}})<q$ and $\tilde{C}_{S_{ld}}\equiv[[\hat{C}^{\intercal}]_{S_{ld}},{\bf\bar{1}}_{q\times 1}]$. Let $S_{re}\doteq V\backslash S_{ld}$, where $V=\{1,...,r\}$. Then, we have ${\rm grank}({\bf col}\{\hat{A},\hat{C}^{S_{ld}}\})\leq{\rm grank}(\hat{A})+{\rm grank}({\hat{C}}_{[S_{ld}]})<r+1-q+q-1=r,$ which means $(\hat{A},\hat{C}^{V\backslash S_{re}})$ is structurally uncontrollable by Lemma 1. Since $|S_{re}|=r-q$, we get $|S_{opt}|\leq|S_{re}|=r-q$.

For the other direction, suppose $|S_{opt}|\leq r-q$. Let ${\cal X}={\cal X}_{1}\cup{\cal X}_{2}$ be the state vertex set of the digraph ${\cal D}(\hat{A},\hat{C})$, with ${\cal X}_{1}=\{x_{1},...,x_{q-1}\}$, ${\cal X}_{2}=\{x_{q},...,x_{r}\}$. Then, it is easy to see that, any two vertices in ${\cal X}_{2}$ are reachable from each other, and every vertex in ${\cal X}_{1}$ is reachable to arbitrary vertex in ${\cal X}_{2}$. Therefore, all vertices in ${\cal X}_{2}$ must be blocked from the corresponding sensors, so that the out-reachability condition is violated. As $|{\cal X}_{2}|=r-q+1>r-q$ and $|S_{opt}|\leq r-q$, to make $(\hat{A},\hat{C}^{V\backslash S_{opt}})$ structurally uncontrollable, it must hold that ${\rm grank}({\bf col}\{\hat{A},\hat{C}^{V\backslash S_{opt}}\})={\rm grank}([\hat{A}^{\intercal},[\hat{C}^{V\backslash S_{opt}}]^{\intercal}])<r$. Again using Lemma 4 on $[\hat{A}^{\intercal},[\hat{C}^{V\backslash S_{opt}}]^{\intercal}]$, we get ${\rm grank}(\hat{C}_{[{V\backslash S_{opt}}]})={\rm grank}(\hat{C}^{V\backslash S_{opt}})<r-(r+1-q)=q-1$. Therefore, ${\rm grank}([[\hat{C}^{\intercal}]_{V\backslash S_{opt}},{\bf\bar{1}}_{|V\backslash S_{opt}|\times 1}])\leq{\rm grank}([\hat{C}^{\intercal}]_{V\backslash S_{opt}})+1<q-1+1=q$. Noting that $[[\hat{C}^{\intercal}]_{V\backslash S_{opt}},{\bf\bar{1}}_{|V\backslash S_{opt}|\times 1}]$ is exactly $\tilde{C}_{V\backslash S_{opt}}$ and $|V\backslash S_{opt}|\geq q$, we therefore obtain $\tilde{C}^{\intercal}$ has $q$ linearly dependent columns indexed by $V\backslash S_{opt}$. Immediately, from Step 1) in the proof of Theorem 3, $\cal G$ has a $k$-clique.

Since the $k$-clique problem is NP-complete, we conclude that determining $|S_{opt}|$ is NP-hard. $\hfill\blacksquare$

VI-B The CCSM Structure of Problem 2

Similar to Section V-A, Problem 2 also has a CCSM structure. To see this, we first resort to the strongly connected component (SCC) decomposition to determine the minimal sensors (denoted by $J^{y}_{re}$) whose removal destroys the output-reachability condition.

A digraph is said to be strongly connected if there is a path from every one of its vertices to the other one. An SCC of $\cal G$ is a subgraph that is strongly connected and no additional edges or vertices from $\cal G$ can be included in this subgraph without breaking its property of being strongly connected. We call an SCC the sink SCC if there is no outgoing edge from this SCC to any other SCCs. Suppose there are $l$ sink SCCs in ${\cal D}(\bar{A})=({\cal X},{\cal E}_{{\cal X},{\cal X}})$. For each sink SCC ${\cal N}_{i}$, denote by $N({\cal N}_{i})$ the out-neighbors of ${\cal N}_{i}$ in ${\cal D}(\bar{A},\bar{C})$ (hence $N({\cal N}_{i})\subseteq{\cal Y}$). Define $J^{y}_{i}=\{j:y_{j}\in N({\cal N}_{i})\}$; that is, $J^{y}_{i}$ is the set of sensors that are out-neighbors of ${\cal N}_{i}$. Then, it is easy to verify that, the minimum number $|J^{y}_{re}|$ of sensors whose removal destroys the output-reachability condition equals $\min\nolimits_{1\leq i\leq l}|J^{y}_{i}|$.

According to Lemma 3 and similar to (1), Problem 2 can be solved by invoking the following problem for at most $O(log_{2}|J^{y}_{re}|)$ times until the objective value is less than $n$.

[TABLE]

Again, (3) belongs to the CCSM problem, as the generic rank of sub-rows of a matrix is submodular on its row set (see [27, Sec 2.3.3]). This indicates that, Problem 2 might be hard to approximate.

VI-C Algorithm with Bounded Matching Deficiencies

We will call the number of vertices of ${\mathcal{R}}(\bar{A})$ that are not matched in any maximum matching of ${\cal B}(\bar{A})=({\cal R}(\bar{A}),{\cal C}(\bar{A}),{\cal E}(\bar{A}))$, as the matching deficiency, which is equal to $n-{\rm grank}(\bar{A})$.

Assumption 1

The matching deficiency of $\bar{A}$ is bounded by some fixed constant $\bar{k}_{s}$ as $n$ and $r$ increase.

Remark 3

Assumption 1 is satisfied by many practical systems. For example, in consensus networks or multi-agent systems [38], every state vertex often has a self-loop, under which circumstance ${\mathcal{B}}(\bar{A})$ has a zero matching deficiency. Moreover, connected ER random networks can be controlled using only one dedicated input with probability $1$ [39], which means the associated matching deficiencies are no more than $1$ [14] with probability $1$. The scenarios where the matching deficiency is bounded by a constant which is neither zero nor one may arise in weakly-connected directed graphs that have a bounded (by a known constant) number of roots or leaves, and graphs with special layered structures, such as being the Cartesian products of factor graphs [40].

To determine the minimal sensors whose removal destroys the matching condition (denoted by $J^{y}_{ma}$), the similar idea to Algorithm 1, by constructing a recursive tree, is utilized. This tree has a similar structure to Fig. 1; see the pseudo code in Algorithm 2. In the $\tau$th layer, each node $\bar{T}_{i}^{(\tau)}$ indexes a subset of sensors such that ${\rm grank}({\bf col}\{\bar{A},\bar{C}_{\bar{T}_{i}^{(\tau)}}\})={\rm grank}(\bar{A})+\tau$, and no other element from $V\backslash\bar{T}_{i}^{(\tau)}$ can be added to $\bar{T}_{i}^{(\tau)}$ without increasing ${\rm grank}({\bf col}\{\bar{A},\bar{C}_{\bar{T}_{i}^{(\tau)}}\})$. Hence, in the $(n-{\rm grank}(\bar{A})-1)$th layer, the leaf with the maximum cardinality is the complementary set to the minimal rows of $\bar{C}$ whose removal from ${\bf col}\{\bar{A},\bar{C}\}$ makes the resulting matrix have generic rank $n-1$. This means the total layers of the recursive tree is exactly $n-{\rm grank}(\bar{A})$.

Theorem 5

Given a system $(\bar{A},\bar{C})$, where $\bar{A}\in\{0,*\}^{n\times n}$, $\bar{C}\in\{0,*\}^{r\times n}$, under Assumption 1, Algorithm 2 provides an optimal solution to Problem 2 with time complexity at most $O((r+n)^{2.373}r^{\bar{k}_{s}})$.

Proof: Based on the above arguments, to prove the optimality, we only need to prove $\Delta$ defined in Line 6 of Algorithm 2 is the maximal subset of rows of $\bar{C}_{V\backslash T_{i}^{\tau}}$ whose addition to ${\bf col}\{\bar{A},\bar{C}_{T_{i}^{\tau}}\}$ can preserve its generic rank. Upon letting $\bar{M}_{S}\doteq{\bf col}\{\bar{A},\bar{C}_{S}\}$, this can be justified from the following observation: for any $S\subseteq V$, $j_{1},j_{2}\in V$, if ${\rm grank}\bar{M}_{S}={\rm grank}\bar{M}_{S\cup\{j_{1}\}}={\rm grank}\bar{M}_{S\cup\{j_{2}\}}$, then ${\rm grank}(\bar{M}_{S\cup\{j_{1},j_{2}\}})\leq{\rm grank}(\bar{M}_{S\cup\{j_{1}\}})+{\rm grank}(\bar{M}_{S\cup\{j_{2}\}})-{\rm grank}(\bar{M}_{S})={\rm grank}(\bar{M}_{S})$, leading to ${\rm grank}(\bar{M}_{S\cup\{j_{1},j_{2}\}})={\rm grank}(\bar{M}_{S})$, where the first inequality is due to the submodularity of the generic rank function of a matrix w.r.t. its rows [27, Sec 2.3.3]. This is similar to the rank-one update property (2).

As for complexity, the SCC decomposition along with the step to obtain $J^{y}_{re}$ has complexity at most $O(n^{2}+nr)$ [29]. In the $i$th layer, there is at most $r^{i}$ nodes, $i=0,...,n-{\rm grank}(\bar{A})-1$. Hence, the number of total nodes in the recursive tree is at most $\sum\nolimits_{\tau=0}^{\bar{k}_{s}-1}r^{\tau}=(r^{\bar{k}_{s}}-1)/(r-1)\to O(r^{\bar{k}_{s}-1})$ ($r\geq 3$). In each node, the operation updating $\bar{T}_{i}^{\tau}$ can be done by calling the maximum matching algorithm on bipartite graphs for at most $r$ times, which incurs complexity $O((r+n)^{2.373}r)$ in the worst case [29]. Hence, the total time complexity of Algorithm 2 is $O(n^{2}+rn+(r+n)^{2.373}r^{\bar{k}_{s}})$, i.e, $O((r+n)^{2.373}r^{\bar{k}_{s}})$. $\hfill\blacksquare$

Remark 4

It can be seen both Line 6 of Algorithms 1 and 2, which result from the rank-one update property of the involved rank function, are crucial to the polynomial time complexity under the corresponding assumptions. However, the similar property does not hold for the general minSBOs. Hence, even under the same constant bound assumptions, it seems unclear whether they can be solved in polynomial time.

VI-D Acceleration of Algorithm 2

The main procedure dominating the complexity of Algorithm 2 is to obtain the minimal sensors $J^{y}_{ma}$ whose removal destroys the matching condition. A method to accelerate this procedure is first to do Dulmage-Mendelsohn decomposition (DM-decomposition) on $\bar{A}$. Then implement Lines 3-15 of Algorithm 2 on the reduced system, which usually has a dimension much less than that of the original system.

Definition 2** (DM-decomposition,[27])**

Let $\bar{M}\in\{0,*\}^{m\times n}$. There exist two permutation matrices $P_{r}\in{\mathbb{R}}^{m\times m}$ and $P_{c}\in{\mathbb{R}}^{n\times n}$, such that

[TABLE]

where $\bar{M}_{12}$, $\bar{M}_{23}$, and $\bar{M}_{34}$ are square with zero-free diagonals. The columns of $\bar{M}_{11}$ (or the rows of $\bar{M}_{44}$) correspond to the unmatched vertices of at least one maximum matching in ${\cal B}(\bar{M})$, and the rows (or columns) of $\bar{M}_{23}$ correspond to the vertices that appear in every maximum matching in ${\cal B}(\bar{M})$. The generic rank of $\bar{M}$ equals $m$ minus the number of rows of $\bar{M}_{44}$.

Using the DM-decomposition, suppose there are two permutation matrices $P_{r}$ and $P_{c}$ with compatible dimensions, such that

[TABLE]

where the left block of the right-hand side matrix of the above-mentioned formula is the DM-decomposition of $\bar{A}^{\intercal}$. Since \left[\begin{array}[]{cc}\bar{A}_{12}&\bar{A}_{13}\\ 0&\bar{A}_{23}\\ \end{array}\right] has full generic rank, it can be validated that

[TABLE]

where ${\mathcal{C}}(\bullet)$ denotes the set of column indices of a matrix. Hence, to determine $J^{y}_{ma}$, we just need to identify the minimum number of columns of ${\bf{col}}\{\bar{B}_{3},\bar{B}_{4}\}$ whose deletion makes \left[{\tiny{{\begin{array}[]{*{20}c}{\bar{A}_{34}}&{\bar{B}_{3}}\\ {\bar{A}_{44}}&{\bar{B}_{4}}\\ \end{array}}}}\right] row generic rank deficient. As ${\bf{col}}\{\bar{B}_{3},\bar{B}_{4}\}$ is obtained by permutating columns of $\bar{C}$, the set of indices of the aforementioned columns exactly correspond to $J^{y}_{ma}$. Hence, the reduced system

[TABLE]

has the same solution as that of the original system $(\bar{A},\bar{C})$ on $J^{y}_{ma}$, where zero matrix is added to $[\bar{A}^{\intercal}_{34},\bar{A}^{\intercal}_{44}]$ to square it. For a sparse matrix $\bar{A}$, $|{\mathcal{C}}([\bar{A}^{\intercal}_{34},\bar{A}^{\intercal}_{44}])|$ is usually much less than $|{\mathcal{C}}(\bar{A})|$. Hence, it is perhaps safe to expect that the DM-decomposition can accelerate Algorithm 2.

VII Numerical Results

We implement some numerical experiments to validate the effectiveness of our proposed algorithms. In our simulations, we generate ER random networks with $n$ nodes and use their adjacent matrices as the state transition matrices, $n$ ranging from $20$ to $120$. For each $n$, the probability of the existence of a directed edge between two nodes is set to be $\frac{c}{n}$, which means that the average degree (the sum of in-degree and out-degree) is $2c$ [29]. The weight of each directed edge is randomly generated in the interval $[-1,1]$. For each $n$, randomly generate $40$ observable systems (to save experiment time, all these systems have maximum geometric multiplicities less than $5$), where $40\%$ of the nodes are randomly selected to be measured by dedicated sensors (i.e., each sensor measures exactly one node). Five methods are adopted to identify a set of sensors whose removal destroys (structural) observability, the first of which is selecting the measured nodes in the descending order of their in-degrees, the second of which is randomly selecting the measured nodes, and the rest three of which are respectively based on Algorithm 1, Algorithm 2 and Algorithm 2 accelerated by the DM-decomposition. The results are shown in Fig. 2.

Fig. 2 shows that for each fixed $n$, the ratio of removed sensors returned by Algorithm 1 is much less than that of the degree based and the random selection methods. This is reasonable, as Algorithm 1 returns the optimal solutions. Moreover, Algorithms 1, 2 and the DM-decomposition accelerated Algorithm 2 return almost the same values for each fixed $n$.777A deep insight shows that for some $n$, Algorithm 1 returns values slightly smaller than those of Algorithm 2. This is consistent with the fact that for a given $(A,C)$, the optimal value of Problem 2 is a theoretical upper bound of that of Problem 1. This indicates that, when the exact values of system matrices are not accessible, the structured system based Algorithm 2 could be an alternative for the observability robustness evaluation. Comparing Fig. 2(a) and (b), it seems that networks with stronger connectivity (i.e., bigger average degrees) tend to have better observability robustness under sensor attacks.

VIII Conclusions

Two problems, namely, the minSRO and the minSBO, are considered in this paper to study the observability robustness under sensor failures. It is shown that all these problems are NP-hard, even restricted to some special cases, both for a numerical system and a structured one, thus confirming the conjecture that verifying the $2s$-sparse observability condition in [3, 4] is NP-hard. The minSROs, both for a numerical system and a structured one, have a CCSM structure that is hard to approximate in general. Nevertheless, under the constant bound assumption on the system eigenvalue geometric multiplicities or the matching deficiencies, by leveraging the rank-one update property of the rank function, polynomial time (in the system dimensions and number of sensors) exact algorithms are given for the minSROs. Numerical experiments on the minSRO show that when we have no access to the exact values of system matrices, the structured system based algorithms could be alternative for the observability robustness evaluation. Currently, we are not able to give (approximation) algorithms for the general minSBOs, which is left for future research.