Persuasion-based Robust Sensor Design Against Attackers with Unknown Control Objectives

TL;DR

This paper develops a robust sensor design framework that uses persuasion strategies to influence attacker beliefs in stochastic control systems, minimizing damage from attackers with unknown objectives.

Contribution

It introduces a linear-plus-noise signaling strategy and a semi-definite programming approach for robust sensor design against unknown attacker objectives in stochastic control.

Findings

Closed-form solution for signaling strategy

Linear matrix inequality condition for belief covariance

Semi-definite program for global optimization

Abstract

In this paper, we introduce a robust sensor design framework to provide "persuasion-based" defense in stochastic control systems against an unknown type attacker with a control objective exclusive to its type. For effective control, such an attacker's actions depend on its belief on the underlying state of the system. We design a robust "linear-plus-noise" signaling strategy to encode sensor outputs in order to shape the attacker's belief in a strategic way and correspondingly to persuade the attacker to take actions that lead to minimum damage with respect to the system's objective. The specific model we adopt is a Gauss-Markov process driven by a controller with a (partially) "unknown" malicious/benign control objective. We seek to defend against the worst possible distribution over control objectives in a robust way under the solution concept of Stackelberg equilibrium, where the sensor is the leader. We show that a necessary and sufficient condition on the covariance matrix of the posterior belief is a certain linear matrix inequality and we provide a closed-form solution for the associated signaling strategy. This enables us to formulate an equivalent tractable problem, indeed a semi-definite program, to compute the robust sensor design strategies "globally" even though the original optimization problem is non-convex and highly nonlinear. We also extend this result to scenarios where the sensor makes noisy or partial measurements. Finally, we analyze the ensuing performance numerically for various scenarios.

Tables

Table 1. TABLE I : Under perfect measurements, i.e., 𝒚 ~ k = 𝒙 ~ k subscript bold-~ 𝒚 𝑘 subscript bold-~ 𝒙 𝑘 \boldsymbol{\tilde{y}}_{k}=\boldsymbol{\tilde{x}}_{k} , comparison of the costs to 𝒫 S subscript 𝒫 𝑆 \mathcal{P}_{S} , i.e., U S subscript 𝑈 𝑆 U_{S} as described in ( 8 ), over various scenarios. Entries at each row corresponds to the cost to 𝒫 S subscript 𝒫 𝑆 \mathcal{P}_{S} for different types of 𝒫 C subscript 𝒫 𝐶 \mathcal{P}_{C} when it constructs the sensor outputs according to the extended type set Ω o superscript Ω 𝑜 \Omega^{o} . And the last column provides the maximum cost of 𝒫 S subscript 𝒫 𝑆 \mathcal{P}_{S} across all types of 𝒫 C subscript 𝒫 𝐶 \mathcal{P}_{C} .

${\Omega }^o$	${\omega }_o$	${\omega }_a$	${\omega }_b$	${\omega }_c$	Max
$\left\{{\omega }_o\right\}$	$43.03$	$323.70$	$354.02$	$352.65$	$354.02$
$\left\{{\omega }_a\right\}$	$122.31$	$191.12$	$351.85$	$246.30$	$351.85$
$\left\{{\omega }_b\right\}$	$119.09$	$320.81$	$185.63$	$267.35$	$320.81$
$\left\{{\omega }_c\right\}$	$106.26$	$202.40$	$224.77$	$137.71$	$224.77$
$\{{\omega }_o,{\omega }_a\}$	$119.65$	$191.12$	$351.11$	$245.95$	$351.11$
$\{{\omega }_o,{\omega }_b\}$	$115.29$	$321.56$	$185.63$	$267.93$	$321.56$
$\{{\omega }_o,{\omega }_c\}$	$106.26$	$202.40$	$224.77$	$137.71$	$224.77$
$\{{\omega }_o,{\omega }_a,{\omega }_b,{\omega }_c\}$	$81.17$	$199.46$	$199.46$	$166.68$	$199.46$

Table 2. TABLE II : Under imperfect measurements, comparison of the costs to 𝒫 S subscript 𝒫 𝑆 \mathcal{P}_{S} , i.e., U S subscript 𝑈 𝑆 U_{S} as described in ( 8 ), over various scenarios. Entries at each row corresponds to the cost to 𝒫 S subscript 𝒫 𝑆 \mathcal{P}_{S} for different types of 𝒫 C subscript 𝒫 𝐶 \mathcal{P}_{C} when it constructs the sensor outputs according to the extended type set Ω o superscript Ω 𝑜 \Omega^{o} . And the last column provides the maximum cost of 𝒫 S subscript 𝒫 𝑆 \mathcal{P}_{S} across all types of 𝒫 C subscript 𝒫 𝐶 \mathcal{P}_{C} .

${\Omega }^o$	${\omega }_o$	${\omega }_a$	${\omega }_b$	${\omega }_c$	Max
$\left\{{\omega }_o\right\}$	$112.39$	$326.21$	$324.82$	$403.09$	$403.09$
$\left\{{\omega }_a\right\}$	$171.64$	$209.99$	$259.51$	$242.26$	$259.56$
$\left\{{\omega }_b\right\}$	$167.80$	$269.60$	$195.90$	$254.20$	$269.60$
$\left\{{\omega }_c\right\}$	$187.26$	$211.62$	$201.94$	$199.51$	$211.62$
$\{{\omega }_o,{\omega }_a\}$	$171.40$	$209.99$	$260.40$	$242.97$	$260.40$
$\{{\omega }_o,{\omega }_b\}$	$167.53$	$270.40$	$195.90$	$255.04$	$270.40$
$\{{\omega }_o,{\omega }_c\}$	$187.26$	$211.62$	$201.94$	$199.51$	$211.62$
$\{{\omega }_o,{\omega }_a,{\omega }_b,{\omega }_c\}$	$185.48$	$210.00$	$210.00$	$202.77$	$210.00$

Equations

$${\mathbf{x}}_{k+1}=A{\mathbf{x}}_k+B{\mathbf{u}}_k+{\mathbf{w}}_k,$$

$${\mathbf{y}}_k=C{\mathbf{x}}_k+{\mathbf{v}}_k,$$

$${\mathbf{s}}_k$$

$${\mathbf{u}}_k={\gamma }_k({\mathbf{s}}_{1:k}),$$

$$\sum _{k=1}^{\kappa }\mathbb{E}\Vert {\mathbf{x}}_{k+1}{\Vert }_Q^2+\mathbb{E}\Vert {\mathbf{u}}_k{\Vert }_R^2,$$

$$\sum _{k=1}^{\kappa }\mathbb{E}\Vert {\mathbf{x}}_{k+1}^{\omega }{\Vert }_{Q^{\omega }}^2+\mathbb{E}\Vert {\mathbf{u}}_k^{\omega }{\Vert }_{R^{\omega }}^2,$$

$$U_C^{\omega }(\eta ,{\gamma }^{\omega }):=\sum _{k=1}^{\kappa }\mathbb{E}\Vert {\mathbf{x}}_{k+1}^{\omega }{\Vert }_{Q^{\omega }}^2+\mathbb{E}\Vert {\mathbf{u}}_k^{\omega }{\Vert }_{R^{\omega }}^2.$$

$$U_S(\eta ,\{{\gamma }^{\omega }{\}}_{\omega \in {\Omega }^o})$$

$$+$$

$$p_{{\omega }_o}\sum _{k=1}^{\kappa }\left(\mathbb{E}\Vert {\mathbf{x}}_{k+1}^{{\omega }_o}{\Vert }_{Q^{{\omega }_o}}^2+\mathbb{E}\Vert {\mathbf{u}}_k^{{\omega }_o}{\Vert }_{R^{{\omega }_o}}^2\right),$$

$$\sum _{\omega \in \Omega }{p}_{\omega }\sum _{k^{\prime }=1}^{\kappa }\mathbb{E}\Vert {\mathbf{x}}_{k^{\prime }+1}^{\omega }{\Vert }_{Q^{{\omega }_o}}^2,$$

$$\mathcal{G}:=\left({\rm Y},\Gamma ,{\mathbf{x}}_1,{\mathbf{w}}_{1:\kappa },{\mathbf{v}}_{1:\kappa }\right)$$

$$\displaystyle\eta\in\operatorname*{argmin\;}_{\eta^{\prime}\in\Upsilon}U_{S}\big{(}\eta^{\prime},\{\gamma^{\omega}(\eta^{\prime})\}_{\omega\in\Omega^{o}}\big{)}$$

$$\displaystyle\gamma^{\omega}(\eta)\in\operatorname*{argmin\;}_{\gamma\in\Gamma}U_{C}^{\omega}\big{(}\eta,\gamma(\eta)\big{)}.$$

$${\mathbf{x}}_{k+1}^o=A{\mathbf{x}}_k^o+{\mathbf{w}}_k,$$

$$\sum _{k=1}^{\kappa }\mathbb{E}\Vert {\mathbf{x}}_{k+1}{\Vert }_{Q^{\omega }}^2+\mathbb{E}\Vert {\mathbf{u}}_k{\Vert }_{R^{\omega }}^2=\sum _{k=1}^{\kappa }\mathbb{E}\Vert {\mathbf{u}}_k^o+K_k^{\omega }{\mathbf{x}}_k^o{\Vert }_{{\Delta }_k^{\omega }}^2+{\delta }_0^{\omega },$$

$$K_k^{\omega }=({\Delta }_k^{\omega }{)}^{-1}{B}^{\prime }{\tilde{Q}}_{k+1}^{\omega }A$$

$${\Delta }_k^{\omega }=B^{\prime }{\tilde{Q}}_{k+1}^{\omega }B+R^{\omega }$$

$${\delta }_0^{\omega }=\mathrm{Tr}\{Q^{\omega }{\Sigma }_1\}+\sum _{k=1}^{\kappa }\mathrm{Tr}\{{\tilde{Q}}_{k+1}^{\omega }{\Sigma }_w\}$$

$${\tilde{Q}}_k^{\omega }=Q^{\omega }+A^{\prime }({\tilde{Q}}_{k+1}^{\omega }-{\tilde{Q}}_{k+1}^{\omega }B({\Delta }_k^{\omega }{)}^{-1}{B}^{\prime }{\tilde{Q}}_{k+1}^{\omega })A$$

$${\mathbf{u}}_k^o={\mathbf{u}}_k+K_k^{\omega }B{\mathbf{u}}_{k-1}+\dots +K_k^{\omega }{A}^{k-2}B{\mathbf{u}}_1.$$

$$L_{k,k}^{\prime }{\mathbf{x}}_k+$$

$$+$$

$$\mathbb{E}\{{\mathbf{x}}_k^o|{\mathbf{s}}_{1:k}\}=\mathbb{E}\{{\mathbf{x}}_k^o|{\mathbf{s}}_{1:k}^o\},$$

$${\mathbf{u}}_k^o=-K_k^{\omega }\mathbb{E}\{{\mathbf{x}}_k^o|{\mathbf{s}}_{1:k}\},$$

$$H_k:=\mathrm{cov}\{\mathbb{E}\{{\mathbf{x}}_k^o|{\mathbf{s}}_{1:k}\}\},$$

$$\underset{\eta \in {\rm Y}}{\min }\underset{p\in \Delta ({\Omega }^o)}{\max }\sum _{\omega \in {\Omega }^o}{p}_{\omega }\left({\xi }^{\omega }+\sum _{k=1}^{\kappa }\mathrm{Tr}\left\{H_k{\Xi }_k^{\omega }\right\}\right),$$

$$H_k=\mathbb{E}\{{\mathbf{x}}_k^o{\mathbf{s}}_{1:k}^{\prime }\}\mathbb{E}\{{\mathbf{s}}_{1:k}{\mathbf{s}}_{1:k}^{\prime }{\}}^{†}\mathbb{E}\{{\mathbf{x}}_k^o{\mathbf{s}}_{1:k}^{\prime }{\}}^{\prime }$$

$${\Sigma }_1^o⪰H_1⪰O,$$

$${\Sigma }_k^o⪰H_k⪰A{H}_{k-1}{A}^{\prime },\text{mbox}fork>1,$$

$${\Sigma }_1^o⪰S_1⪰O,$$

$${\Sigma }_k^o⪰S_k⪰A{S}_{k-1}{A}^{\prime },\text{mbox}fork>1,$$

$${\Sigma }_k^o-A{S}_{k-1}{A}^{\prime }={\bar{U}}_k\left[\begin{array}{cc}{\bar{\Lambda }}_k& O\\ O& O\end{array}\right]{\bar{U}}_k^{\prime }$$

$$T_k:=\left[\begin{array}{cc}{\bar{\Lambda }}_k^{1/2}& O\end{array}\right]{\bar{U}}_k^{\prime }(S_k-A{S}_{k-1}{A}^{\prime }){\bar{U}}_k\left[\begin{array}{c}{\bar{\Lambda }}_k^{1/2}\\ O\end{array}\right]$$

Full text

I Introduction

Cyber connectedness of control systems has brought in new security challenges where attackers can manipulate control systems at unprecedented levels with various malicious tasks on their agenda[1, 2, 3]. We can view such intelligent attackers as decision makers that take actions driven by and compatible with their own objective and information available to them. This implies that the kind of information available to an attacker has an indirect impact on what actions the attacker would take. Correspondingly, it is intuitively expected that if we can manipulate the information available to attackers, then we can persuade them to attack the system in a way in line with the system’s objective to the extent possible, so that the attack would cause minimum damage to the system.

There are certain distinct challenges for persuasion-based defense measures. For example, attackers make their decisions based on the belief they have formed using the information available to them. A challenge is how to shape that belief in a desired way by controlling the information available to the attacker. The first requirement (and challenge) here is to be able to identify how an attacker would form its belief. However, when an attacker forms its belief strategically, there might be multiple Nash equilibria in a general-sum setting111In a zero-sum setting, there exists a unique “babbling equilibrium” where the attacker forms a belief independent of the information provided by a defender (since the defender would not share anything useful) whereas the defender just makes irrelevant information available to the attacker (since a more informed attacker would not cause less damage to the system)., as shown in the strategic information transmission framework [4]. In that case how an attacker forms its belief is not well defined since the attacker might be forming its belief differently at different equilibria and whether (and which) equilibrium will be realized would not be known, and constitutes an uncertainty. Furthermore, even when there exists a well-defined characterization of how an attacker would form its belief, another challenge is what decision the attacker would make based on that belief, since that decision would depend on the attacker’s malicious objective. This implies that persuasion-based defense is attack-specific. Therefore another challenge is how to persuade an attacker whose objective is not known to act in a certain way.

Before addressing these challenges, let us briefly review the literature from a broader perspective where a decision maker seeks to induce another one to take certain actions. A closely related framework is Bayesian persuasion, introduced by Kamenica and Gentzkow in their seminal paper [5]. They addressed how a sender could persuade a receiver to take certain actions under the solution concept of Stackelberg equilibrium [6] where the sender (the receiver) is the leader (the follower). In other words, the receiver is aware of the sender’s signaling strategy while taking its actions. Such a leader-follower scheme creates an environment where the receiver’s actions are not strategic, and therefore there is a well-defined characterization for how the belief is formed.

In Bayesian persuasion framework, a necessary and sufficient condition is that mean of posterior belief must be equal to the prior belief. This enables us to formulate an equivalent problem over distributions over posterior beliefs under a linear equality constraint. The authors provided a geometrical interpretation to compute the solution, which necessitates computation of a convex envelope of a function. Although this interpretation provides essential insight for designing signaling strategies and has led to various applications (see the recent survey on Bayesian persuasion [7] and the references therein), it is viable only for scenarios where the state space is very small.

We emphasize that [5] studies the Bayesian persuasion problem in a very general framework, where the underlying distribution is arbitrary as long as its support set is compact, cost functions are arbitrary, and the signaling strategy is any stochastic kernel (between the state space and the signal space). Therefore if we consider specific distributions and cost functions, we should be able to obtain more tractable solutions. For example in [8], the author addressed Bayesian persuasion problem for multi-variate Gaussian information and quadratic cost functions, and provided a closed-form solution for the optimal signaling strategy, which turns out to be a linear function. We note that the studies [5, 8] focused on static systems. To be able to adopt this framework to control systems, an important step would be to extend the framework to dynamic systems. In [9], we have extended the results in [8] to dynamic environments where the underlying information is a discrete-time Gauss Markov process. We showed that there exists a linear signaling strategy optimal within the general class of measurable policies and provided a semi-definite program (SDP) to compute optimal signaling strategies numerically.

Before delving into persuasion-based defense measures in control systems, let us also review the literature on security of control systems. To this end, we selectively focus on studies where an attacker can monitor and intervene the links from sensor to controller and from controller to actuator so that there would be an information flow to the attacker who monitors these links. In [10], the authors showed that an attacker can lead to unbounded estimation error by injecting false data into the link from sensor to controller when there exists a certain threshold-based detector monitoring the link. In [11, 12], the authors characterized the reachable set to which an evasive attacker can drive the system by injecting data into both links jointly. In [13, 14], the authors analyzed attacks where an attacker seeks to drive the state of the system according to his/her adversarial goal evasively by injecting data into both links jointly. In [15], the authors have analyzed optimal attack strategies to maximize quadratic cost of a system with linear Gaussian dynamics by injecting false data into the link from controller to actuator without being detected.

As seen in the literature reviewed above, an attacker can monitor and intervene the links in a control system, e.g., as illustrated in Fig. 1 for a linear quadratic Gaussian (LQG) control system. This implies that an attacker can bypass the controller of a system by monitoring and intervening both links jointly. Then the attacker would receive the sensor outputs and could generate its own control inputs to conduct its malicious control objective similar to the controller of the system. From this viewpoint, within the Bayesian persuasion framework, we can encode the sensor outputs to persuade the attacker to generate control inputs that would minimize the system’s very own control objective as much as possible. We partially addressed this challenge in [9, 16, 17]. In [9], we formulated an optimal linear signaling rule in an LQG setting when attacker’s control objective is known to the sensor. This limits applicability of the solution since the sensor must know when there is an attack and what the control objective of the attacker is beforehand. In the follow-up papers [16, 17], we showed that this limitation could be relaxed in a straightforward way if the sensor knows the distribution over the control objectives of attackers that may attack the system and at what probability there might be an attack. Correspondingly, a risk-neutral sensor could minimize the expected damage with respect to that distribution. However, this still limits its applicability since the sensor must know the underlying distribution over the attack space.

Although it is not a persuasion-based defense, it is worth noting that in [18], the authors proposed linear encoding schemes222We use the terminologies encoding scheme and signaling rule/strategy interchangeably. for sensor outputs in an LQG control problem in order to enhance detectability of false data injection attacks under an essential assumption that the encoding matrix is oblivious to attackers. In that respect, the encoding scheme could be viewed as corrupting the information available to attackers (without impacting the information flowing to the controller of the system) in order to limit the attacker’s capability to evade detection rather than persuading the attacker to attack in a certain way. And this security measure becomes undermined completely once the encoding scheme is compromised.

Coming to the specifics of this paper, we also consider a discrete-time LQG control problem, similar to the studies reviewed above [13, 14, 15, 16, 17, 18], but with important differences. We particularly seek to design linear-plus-noise signaling rules for persuasion-based robust sensor design against an unknown type attacker with a control objective exclusive to its type. It is robust in the sense that sensor outputs are designed against the worst possible distribution over all possible control objectives of the attacker. We consider the scenario where the set of types is finite and known to the system. Under the solution concept of Stackelberg equilibrium, we consider the setting where the sensor is the leader and the attacker is the follower. This yields that the underlying encoding scheme is not necessarily oblivious to the attacker. Furthermore, we address the scenarios where the sensor can have partial or noisy measurements different from the models in [9, 16, 17]. Note that the worst case distribution over the type set turns out to be not necessarily a degenerate distribution, i.e., defending only against the (strongest) type of attack that leads to largest cost for the system is not necessarily optimal with respect to the system’s objective.

Due to the underlying leader-follower setting, the response of the follower (the attacker) is non-strategic and the problem faced by the leader (the sensor) turns out to be an optimization problem rather than a fixed-point problem. By using the classical method of completion of squares [19], we can show that the objective function in that optimization problem is linear in the covariance matrix of the posterior estimate of the underlying (control-free) state. However that covariance matrix depends on the encoding scheme in a nonlinear way, which leads to a non-convex and highly nonlinear optimization problem, to be solved globally in order to compute robust sensor design strategies. In [9], our previous inspection revealed a necessary condition on this covariance matrix, which is just a linear matrix inequality. Here we show that this necessary condition is also a sufficient condition when the sensor selects linear plus noise signaling strategies. Furthermore for any matrix satisfying the necessary condition, we provide a closed-form solution for the associated linear plus noise signaling rule. Therefore instead of trying to solve a non-convex and highly nonlinear optimization problem, we are able to bring the problem to one of solving a linear optimization problem under linear matrix inequality constraints, which can be done numerically using existing SDP solvers effectively, e.g., [20, 21]. This solution concept can be seen to have similar flavor with our previous paper [9], reviewed above. However, here we develop and present a completely new and more comprehensive set of technical tools since the results of [9] cannot be adopted for the general setting of this paper. The reader can refer to Appendix A for a detailed discussion on this matter.

We now highlight the main contributions of this paper as follows:

• •

We show that a necessary and sufficient condition on the covariance matrix of the posterior estimate of control-free state is a certain linear matrix inequality. This result is important by itself since it yields a tractable solution concept to design sensor outputs in general settings not limited to the special setting studied in this paper.

• •

Based on this necessary and sufficient condition, we provide an SDP equivalent to the original optimization problem faced by the sensor, which is non-convex and highly nonlinear, while the SDP could be solved globally using existing powerful computational tools effectively [20, 21]. Furthermore, this result can be extended to scenarios where there are partial or noisy measurements.

• •

We note that robust signaling rule can dictate sensors to introduce irrelevant information into sensor outputs quite contrary to other settings, e.g., when there is no uncertainty (or there is imperfect information) on attacker’s type. This observation enables us to draw the following conclusions:

• –

The equivalence result does not necessarily hold if the sensor can only use linear signaling rules.

• –

Based on Blackwell’s Irrelevant Information Theorem [22, Theorem D.1.1], linear signaling strategies are not the best one within the general class of measurable strategies in this robust setting.

The paper is organized as follows: In Section II, we formulate the robust sensor design game. In Section III, we analyze the equilibrium of the robust sensor design game under perfect measurements. In Section IV, we extend the results to the cases where there are partial or noisy measurements. In Section V, we examine numerically the performance of the proposed scheme for various scenarios. We conclude the paper in Section VI with several remarks and possible research directions. An appendix provides further discussion on related literature, proofs of all technical results in the order they appear throughout the paper, and some closed-form expressions for the reader’s reference.

Notation: We denote a collection of parameters via a subscript, e.g., $\{x_{k}\}$, by dropping the subscript, e.g., $x$, for notational brevity. For a vector $x$ and a matrix $A$, $x^{\prime}$ and $A^{\prime}$ denote their transposes, and $\|x\|$ denotes the Euclidean ($L^{2}$) norm of the vector $x$. For a matrix $A$, $\mathrm{Tr}\{A\}$ and $\mathrm{rank}\{A\}$ denote its trace and rank, respectively. We denote the identity and zero matrices with the associated dimensions by $I$ and $O$, respectively. We denote the set of $m$-by-$m$ symmetric, positive semi-definite, and positive definite matrices by $\mathbb{S}^{m}$, $\mathbb{S}_{+}^{m}$, and $\mathbb{S}_{++}^{m}$, respectively. For a matrix $A$, $A^{\dagger}$ denotes its Moore-Penrose inverse. For positive semi-definite matrices $A$ and $B$, $A\succeq B$ means that $A-B$ is also a positive semi-definite matrix. We denote the Kronecker product of matrices $A$ and $B$ by $A\otimes B$.

We denote an ordered set $\{x_{k},\ldots,x_{1}\}$ and its augmented vector version, $\begin{bmatrix}x_{k}^{\prime}&\cdots&x_{1}^{\prime}\end{bmatrix}^{\prime}$, by $x_{1:k}$, by some abuse of notation. $\mathcal{N}(0,.)$ denotes the multivariate Gaussian distribution with zero mean and designated covariance matrix. We denote random variables by bold lower case letters, e.g., $\boldsymbol{x}$. We denote expectation and (co)variance of a random variable $\boldsymbol{x}$ by $\mathbb{E}\{\boldsymbol{x}\}$ and $\mathrm{cov}\{\boldsymbol{x}\}$, respectively. For random variables $\boldsymbol{x}$ and $\boldsymbol{y}$, $\mathbb{E}\{\boldsymbol{x}|\boldsymbol{y}\}$ denotes the expectation of $\boldsymbol{x}$ with respect to the random variable $\boldsymbol{y}$. We denote the set of all possible probability distributions over a set $\Omega$ by $\Delta(\Omega)$.

II Problem Formulation

Consider a control system, as illustrated in Fig. 1, whose underlying state dynamics and sensor measurements are described, respectively, by:

[TABLE]

for $k=1,\ldots,\kappa$, where333Even though we consider time-invariant matrices $A,B$, and $C$, for notational simplicity, the provided results could be extended to time-variant cases rather routinely. Furthermore, we consider all the random parameters to have zero mean; however, the derivations can be extended to non-zero mean case in a straight-forward way. $A\in\mathbb{R}^{m\times m},B\in\mathbb{R}^{m\times r}$, and $C\in\mathbb{R}^{n\times m}$, and the initial state $\boldsymbol{x}_{1}\sim\mathcal{N}(0,\Sigma_{1})$. The additive state and measurement noise sequences $\{\boldsymbol{w}_{k}\}$ and $\{\boldsymbol{v}_{k}\}$, respectively, are white Gaussian vector processes, i.e., $\boldsymbol{w}_{k}\sim\mathcal{N}(0,\Sigma_{w})$ and $\boldsymbol{v}_{k}\sim\mathcal{N}(0,\Sigma_{v})$; and are independent of the initial state $\boldsymbol{x}_{1}$ and of each other.

As seen in Fig. 1, measurements $\boldsymbol{y}_{1:k}\in\mathbb{R}^{nk}$ are encoded into a signal $\boldsymbol{s}_{k}\in\mathbb{R}^{m}$ through a signaling strategy $\eta_{k}(\cdot)$, which is a stochastic kernel, and $\boldsymbol{s}_{k}=\eta_{k}(\boldsymbol{y}_{1:k})$ almost everywhere over $\mathbb{R}^{m}$. We specifically consider “linear plus noise” signaling rules such that the signal $\boldsymbol{s}_{k}$ is given by

[TABLE]

almost everywhere over $\mathbb{R}^{m}$, where $L_{k}\in\mathbb{R}^{nk\times m}$ can be any $nk$-by-$m$ deterministic matrix, and $\boldsymbol{\vartheta}_{k}\sim\mathcal{N}(0,\Theta_{k})$ is a zero mean multivariate Gaussian noise independent of every other parameter, and its covariance matrix $\Theta_{k}\in\mathbb{S}^{m}_{+}$ can be any $m$-by-$m$ positive semi-definite matrix. We denote the set of such signaling rules by $\Upsilon_{k}$, i.e., $\eta_{k}\in\Upsilon_{k}$. Furthermore, the closed-loop control input $\boldsymbol{u}_{k}\in\mathbb{R}^{r}$ is given by

[TABLE]

almost everywhere over $\mathbb{R}^{r}$, where $\gamma_{k}(\cdot)$ can be any Borel measurable function from $\mathbb{R}^{mk}$ to $\mathbb{R}^{r}$. We denote the set of such control policies by $\Gamma_{k}$, i.e., $\gamma_{k}\in\Gamma_{k}$. For notational brevity, let us denote signaling (control) strategies and the associated sets across the horizon by $\eta$ and $\Upsilon$ ($\gamma$ and $\Gamma$), respectively.

II-A Defense Model

We consider an LQG control problem, where the controller of the system selects a measurable control strategy $\gamma\in\Gamma$ in order to minimize

[TABLE]

where444For notational simplicity, we consider time-invariant $Q$ and $R$. However, the results provided could be extended to the general time-variant case rather routinely. $Q\in\mathbb{S}^{m}_{+}$ and $R\in\mathbb{S}_{++}^{r}$. As illustrated in Fig. 1, we consider the encoder of the system as a decision maker, denoted by $\mathcal{P}_{S}$. $\mathcal{P}_{S}$ selects the signaling strategy $\eta\in\Upsilon$ in order to minimize the same objective with the controller, (5). Note that if there were no attacks, identity function, where the measurements are shared with the controller fully, would be an optimal encoding scheme due to the data processing inequality [23, Theorem 2.8.1]. However, we consider here the scenarios where there can be an attack with an unknown control objective. Correspondingly there might be encoding schemes that do not share the measurements fully and can lead to better performance with respect to (5).

II-B Attack Model

We consider an attacker who is aware of the underlying state dynamics, i.e., gain matrices $A$, $B$, and $C$; covariance matrices $\Sigma_{1}$, $\Sigma_{w}$, and $\Sigma_{v}$; and the encoding scheme, i.e., $\eta$. The attacker is of an unknown type, which determines its control objective. Let us denote the set of all possible types by $\Omega$. We suppose that $\Omega$ is finite and known by the system. We seek to provide a compact and unified analysis. Therefore we consider that the control objective of type $\omega\in\Omega$ is given by

[TABLE]

where $Q^{\omega}\in\mathbb{S}_{+}^{m}$ and $R^{\omega}\in\mathbb{S}_{++}^{r}$ are exclusive to the type $\omega$. Note that when there is an attack, the attacker selects a control strategy $\gamma^{\omega}\in\Gamma$ in order to construct its control input $\boldsymbol{u}_{k}^{\omega}$ and correspondingly its control objective includes the term $\mathbb{E}\|\boldsymbol{u}_{k}^{\omega}\|_{R^{\omega}}^{2}$. We also denote the state driven by type-$\omega$ attacker by $\boldsymbol{x}_{k}^{\omega}$, to make it explicit.

Remark 1* (Versatility of Control Objectives).*

We model the control objectives of the system and the attacker by (5) and (6), respectively, within a unified framework. However, arbitrariness of weight matrices in the control objectives (5) and (6) brings in flexibility to model various attack scenarios (that are not in the exact form of (6)) through the transformation of the underlying state space as exemplified in Section V. $\square$

II-C Game Model

We analyze the interaction between the attacker and $\mathcal{P}_{S}$ under the solution concept of Stackelberg equilibrium where $\mathcal{P}_{S}$ is the leader. Note that from the viewpoint of $\mathcal{P}_{S}$, either the controller of the system is receiving the sensor output and driving the state or there is an unknown type attack, and it is getting the sensor output and it is generating the control input. Since whether there is an attack or not is also an uncertainty, let us view the attacker and the controller of the system as a single player in a unified way, denoted by $\mathcal{P}_{C}$, with an unknown type from the extended type set $\Omega^{o}:=\Omega\cup\{\omega_{o}\}$, where type-$\omega_{o}$ corresponds to the controller of the system, i.e., $Q^{\omega_{o}}=Q$ and $R^{\omega_{o}}=R$. Therefore, we consider a Stackelberg game between the leader $\mathcal{P}_{S}$ and the follower $\mathcal{P}_{C}$ (of an unknown type).

We note that depending on its type, $\mathcal{P}_{C}$ selects different control policies, which lead to different control inputs, and states. Therefore for type-$\omega$ $\mathcal{P}_{C}$, we use $\gamma_{k}^{\omega}$, $\boldsymbol{x}_{k}^{\omega}$ and $\boldsymbol{u}_{k}^{\omega}$. The objective of type-$\omega$ $\mathcal{P}_{C}$ is given by

[TABLE]

On the other hand, the objective of $\mathcal{P}_{S}$ is given by

[TABLE]

where $\Delta(\Omega^{o})$ denotes all possible distributions over the extended type set $\Omega^{o}$. Note that the maximization in (8) computes the cost of $\mathcal{P}_{S}$ for the worst case distribution over $\Omega^{o}$.

Before describing the game formally, let us take a closer look at $\mathcal{P}_{S}$’s objective (8). We can view (8) consisting of two parts, one of which is

[TABLE]

where $p_{\omega_{o}}$ corresponds to the probability that the controller of the system drives the state under the worst case distribution and the summation is identical to (5) since $Q^{\omega_{o}}=Q$ and $R^{\omega_{o}}=R$. The other part is

[TABLE]

which implies that $\mathcal{P}_{S}$ seeks to minimize $\sum_{k=1}^{\kappa}\mathbb{E}\|\boldsymbol{x}_{k+1}^{\omega}\|_{Q^{\omega_{o}}}^{2}$ when type-$\omega$ attacker drives the state. Note that it includes only the first term in (5) since we consider that $\mathcal{P}_{S}$ would not necessarily want the attacker to have small size control inputs.

Definition* (Robust Sensor Design Game).*

The robust sensor design game

[TABLE]

is a Stackelberg game between the leader $\mathcal{P}_{S}$ and the follower $\mathcal{P}_{C}$. Objectives of $\mathcal{P}_{C}$ and $\mathcal{P}_{S}$ are given by (7) and (8), respectively. Then a tuple of strategies $(\eta,\{\gamma^{\omega}\}_{\omega\in\Omega^{o}})$ attains the Stackelberg equilibrium provided that

[TABLE]

where, by an abuse of notation, we denote type-$\omega$ $\mathcal{P}_{C}$’s strategy $\gamma^{\omega}$ by $\gamma^{\omega}(\eta)$ to show its dependence on $\mathcal{P}_{S}$’s signaling rules due to the leader-follower scheme, explicitly.

Note that there might be multiple best responses by $\mathcal{P}_{C}$ for a signaling strategy. Correspondingly (12a) would not be well defined if these best responses lead to different costs for $\mathcal{P}_{S}$. However, as we will show later in detail, reaction set of type-$\omega$ $\mathcal{P}_{C}$ turns out to be an equivalence class such that all $\gamma^{\omega}$ in the reaction set lead to the same control input almost surely, and correspondingly lead to the same cost for $\mathcal{P}_{S}$.

III Robust Sensor Design Framework

In this section, we consider the special case where $\mathcal{P}_{S}$ has access to perfect measurements, i.e., $\boldsymbol{y}_{k}=\boldsymbol{x}_{k}$ for $k=1,\ldots,\kappa$; the general noisy/partial measurements case will be addressed later in Section IV. To compute the equilibrium of the game $\mathcal{G}$, we first focus on best response strategy of $\mathcal{P}_{C}$ for a given signaling strategy. This enables us to formulate the optimization problem faced by $\mathcal{P}_{S}$ to compute robust signaling strategies. Even though this is a finite-dimensional optimization problem, further inspection reveals that it is highly nonlinear and non-convex. Therefore a generic approach would not be able to address it globally. To mitigate this issue, we formulate a tractable problem equivalent to the original optimization problem. We can solve this tractable problem globally using existing computational tools effectively. Given that solution, we also show how we can compute the associated signaling strategies. We now provide the details of these steps.

An important challenge in the design of encoding schemes in control systems compared to communication systems is that the underlying state depends on control inputs. To mitigate this issue, we introduce control-free state $\{\boldsymbol{x}_{k}^{o}\}$ evolving according to

[TABLE]

and $\boldsymbol{x}_{1}^{o}=\boldsymbol{x}_{1}$. As shown in [24], the routine technique of completion of squares yields that

[TABLE]

where the matrices $K_{k}^{\omega}$, $\Delta_{k}^{\omega}$, and scalar $\delta_{o}^{\omega}$ are given by

[TABLE]

and $\{\tilde{Q}_{k}^{\omega}\}$ satisfies the following discrete-time dynamic Riccati equation

[TABLE]

and $\tilde{Q}_{\kappa+1}^{\omega}=Q^{\omega}$. Furthermore $\boldsymbol{u}_{k}^{o}$ depends on the control inputs through the following transformation:

[TABLE]

Note that $\Delta_{k}^{\omega}$ is positive definite for all $k$.

Contrary to team problems (where all decision makers have the same objective), as studied in [24], in non cooperative settings, (14) does not imply that a control input leading to $\boldsymbol{u}_{k}^{o}=-K_{k}^{\omega}\mathbb{E}\{\boldsymbol{x}_{k}^{o}|\boldsymbol{s}_{1:k}\}$ is optimal since control inputs can have an impact on the signals that will be generated in future stages. However, as we will show below, $\mathcal{P}_{C}$ cannot impact the signals generated in future stages when $\mathcal{P}_{S}$ uses linear plus noise signaling strategies only. To show this, we let the gain matrix $L_{k}\in\mathbb{R}^{nk\times m}$ in signaling strategy $\eta_{k}$, as described in (3), be partitioned as $L_{k}=\begin{bmatrix}L_{k,k}^{\prime}&\cdots&L_{k,1}^{\prime}\end{bmatrix}^{\prime}$, where $L_{k,j}\in\mathbb{R}^{n\times m}$. Then, signal $\boldsymbol{s}_{k}$ can be written as

[TABLE]

where the term in-between $\{\cdot\}$ is $\sigma$-$\boldsymbol{s}_{1:k-1}$ measurable. Similar to555We note that [9, Lemma 12] shows a result similar to (16) when the sender selects only linear and memoryless signaling strategies. [9, Lemma 12], this yields that

[TABLE]

where we define $\boldsymbol{s}_{k}^{o}:=L_{k}^{\prime}\boldsymbol{x}^{o}_{1:k}+\boldsymbol{\vartheta}_{k}$. Correspondingly, since $\Delta_{k}^{\omega}$ is positive definite for all $k$, right-hand sides of (14) and (15) yield that optimal reaction of type-$\omega$ $\mathcal{P}_{C}$ is indeed the one that leads to

[TABLE]

almost everywhere over $\mathbb{R}^{r}$, since $\mathbb{E}\{\boldsymbol{x}_{k}^{o}|\boldsymbol{s}_{1:k}\}$ does not depend on $\boldsymbol{u}_{1:k}$. This shows that all strategies in the best reaction set of type-$\omega$ $\mathcal{P}_{C}$ lead to (16) and therefore lead to the same cost for $\mathcal{P}_{S}$.

Based on (16), the following lemma shows that we can write the optimization objective in (8) as a linear function of the covariance matrix of the posterior estimate of control-free state, i.e.,666Henceforth we say “covariance matrix” instead of “covariance matrix of posterior estimate of control-free state”, and we say “posterior” instead of “posterior estimate of control-free state”.

[TABLE]

for $k=1,\ldots,\kappa$.

Lemma 1**.**

The problem faced by $\mathcal{P}_{S}$, i.e., (12a), can be written as

[TABLE]

where $\Xi_{k}^{\omega}$ is a certain symmetric matrix, described in (50), that does not depend on the optimization arguments, and $\xi^{\omega}\in\mathbb{R}$, described in (44) and (49), is a certain fixed scalar.

We emphasize that complexity of the objective functions (5) and (8) is buried in fixed parameters $\{\{\Xi_{k}^{\omega}\},\xi^{\omega}\}$. Based on this observation, we make the following remarks:

Remark 2* (Versatility of the results with respect to $\mathcal{P}_{C}$’s objective).*

The problem faced by $\mathcal{P}_{S}$ is a linear function of the covariance matrices $\{H_{k}\}$ since optimal reaction of $\mathcal{P}_{C}$ turns out to be linear in the posterior estimate of the control-free state, i.e., $\mathbb{E}\{\boldsymbol{x}_{k}^{o}|\boldsymbol{s}_{1:k}\}$, as seen in (16). Therefore the results henceforth hold for any other scenarios where $\mathcal{P}_{C}$ has any other objective in which optimal reaction of $\mathcal{P}_{C}$ still turns out to be a linear function of the posterior estimate. Note that we need to compute the associated $\{\{\Xi^{\omega}_{k}\},\xi^{\omega}\}$ accordingly. $\square$

Remark 3* (Versatility of the results with respect to $\mathcal{P}_{S}$’s objective).*

We motivate and consider the case where $\mathcal{P}_{S}$ has objective (8). However, the results henceforth would also hold for scenarios where $\mathcal{P}_{S}$’s objective is any other (convex or non-convex) quadratic function of the state and control input. Note that we also need to compute the associated $\{\{\Xi^{\omega}_{k}\},\xi^{\omega}\}$ accordingly. $\square$

Compared to original form of the optimization problem (12a), the optimization problem (17) has a concrete structure showing that the optimization function depends on the signaling strategy through the covariance matrix only, and it is a linear function of the covariance matrix. Therefore it is instructive to study the relationship between the covariance matrix and the signaling strategy. Since the underlying distributions are all jointly Gaussian, we can express the covariance matrix in closed form:

[TABLE]

and the signal $\boldsymbol{s}_{k}$ is given by (3). This yields that even though computing robust signaling strategies would mean finding a certain number of matrices, it is a highly nonlinear and non-convex optimization problem due to the matrix inversion in (18). Therefore it is difficult to obtain a global solution through a generic attempt, e.g., genetic algorithm [25] or particle swarm optimization [26]. On the other hand, the following proposition shows that there is a computationally tractable relationship between the covariance matrix and linear plus noise signaling strategies.

Proposition 1** (A Necessary and Sufficient Condition).**

For any signaling rule $\eta\in\Upsilon$, covariance matrix of posterior estimate of the control-free state, $\{H_{k}=\mathrm{cov}\{\mathbb{E}\{\boldsymbol{x}_{k}^{o}|\boldsymbol{s}_{1:k}\}\}\}$, satisfies

[TABLE]

where777By the definition of $\Sigma_{k}^{o}$, we have $\Sigma_{k}^{o}=A\Sigma_{k}^{o}A^{\prime}+\Sigma_{w}$. $\Sigma_{k}^{o}:=\mathbb{E}\{\boldsymbol{x}_{k}^{o}(\boldsymbol{x}_{k}^{o})^{\prime}\}$.

Furthermore for any collection of positive semi-definite matrices $S_{1:\kappa}$ satisfying

[TABLE]

there exists a (memoryless) linear-plus-noise signaling strategy888Such a signaling strategy is described in (21) later. $\eta\in\Upsilon$ such that $S_{k}=H_{k}$.

In the following, we provide a description of signaling strategies999A derivation of the associated signaling strategies can be found in the constructive proof of Proposition 1, which is provided in Appendix C. that lead to covariance matrices matching with a given collection of positive semi-definite matrices $S_{1:\kappa}$ satisfying (35). To this end, we let

[TABLE]

be the eigen-decomposition such that $\bar{\Lambda}_{k}\in\mathbb{S}_{++}^{t_{k}}$, where $t_{k}=\mathrm{rank}\{\Sigma_{k}^{o}-AS_{k-1}A^{\prime}\}$. Furthermore, we let

[TABLE]

have the eigen-decomposition $T_{k}=U_{k}\Lambda_{k}U_{k}^{\prime}$ with eigenvalues, e.g., $\lambda_{k,1},\ldots,\lambda_{k,t_{k}}$. We note that $\lambda_{k,t}$ turns out to be in $[0,1]$. Then, the associated signaling strategy is given by

[TABLE]

where $\boldsymbol{\vartheta}_{k}\sim\mathcal{N}(0,\Theta_{k})$ with $\Theta_{k}=\mathrm{diag}\{\theta_{k,1}^{2},\ldots,\theta_{k,t_{k}}^{2},0,\ldots,0\}$, and the gain matrix $L_{k,k}$ is given by

[TABLE]

where $\Lambda_{k}^{o}:=\mathrm{diag}\{\lambda_{1,1}^{o},\ldots,\lambda_{1,t_{k}}^{o}\}$ is a diagonal matrix such that $\{\lambda_{k,i}^{o},\theta_{k,i}^{2}\}_{i=1}^{t_{k}}$ satisfies

[TABLE]

The following corollary to Proposition 1 provides a problem equivalent to (17).

Corollary 1** (Equivalence Result).**

The problem faced by $\mathcal{P}_{S}$, i.e., (8), is equivalent to

[TABLE]

subject to the following linear matrix inequalities:

[TABLE]

And given a solution $S_{1:\kappa}$, an associated signaling strategy can be computed according to (21).

Henceforth, we will be working with (22) instead of (17) while analyzing the equilibrium of the game $\mathcal{G}$. Furthermore, for brevity of presentation, let us introduce

[TABLE]

and $\Psi\subset\mathbb{S}_{+}^{m\kappa}$ be the set corresponding to the constraints (23) in this new high-dimensional space, i.e., $\mathbb{R}^{m\kappa\times m\kappa}$. With this new notation, the problem faced by $\mathcal{P}_{S}$, i.e., (22), can be written as

[TABLE]

The following proposition addresses the existence of an equilibrium for the game $\mathcal{G}$.

Proposition 2** (Existence Result).**

There exists at least one tuple of strategies $(\eta,\{\gamma^{\omega}(\eta)\}_{\omega\in\Omega^{o}})$ attaining the equilibrium of the Stackelberg game $\mathcal{G}$, i.e., satisfying (12).

It is instructive to examine whether optimal signaling strategies end up using irrelevant information or not. The inner optimization problem in (24)

[TABLE]

is a convex function of $S\in\Psi$ since the maximum of any family of linear/affine functions is a convex function [27]. Therefore, there might be examples where any extreme point of the constraint set $\Psi$ is not a solution for (24). Correspondingly, optimal signaling strategy can turn out to be including irrelevant information. This is interesting in view of Blackwell’s Irrelevant Information Theorem [22, Theorem D.1.1]. Particularly, the theorem says that given a cost measure, for any Borel measurable function that uses irrelevant information, there exists another Borel measurable function that does not use any irrelevant information and can lead to a cost less than or equal to the one attained with the former function. Therefore we can conclude that in this robust setting, linear signaling strategies are not the best one within the general class of measurable strategies.

Next, we seek to compute the equilibrium of $\mathcal{G}$. To this end, we examine the equilibrium conditions further. In particular, according to (24), given $S\in\Psi$, maximizing $p^{*}\in\Delta(\Omega^{o})$ is given by

[TABLE]

since the optimization objective in (24) is a linear function of $p\in\Delta(\Omega^{o})$. Based on the observation (26) and the assumption that the type set $\Omega^{o}$ is finite, the following theorem provides an algorithm to compute robust sensor outputs.

Theorem 1** (Computing the Equilibrium).**

The value of the Stackelberg equilibrium (24) is given by $\mu=\min_{\omega\in\Omega^{o}}\{\mu_{\omega}\}$, where

[TABLE]

Furthermore, let $\mu_{\omega^{*}}=\mu$ and

[TABLE]

Then, given $S^{*}\in\Psi$, an associated linear-plus-noise signaling strategy $\eta\in\Upsilon$ can be computed according to (21).

For the reader’s reference, in the following, we list the steps to compute robust sensor design strategies:

• •

We first compute the matrices $\Xi_{k}^{\omega}$, described in (50), and scalars $\xi^{\omega}$, described in (44) and (49), for each type of control objectives, $\omega\in\Omega^{o}$. This step includes computation of gain matrices of optimal control input, e.g., (16), in an LQG control problem.

• •

Given computed $\{\{\Xi_{k}^{\omega},\xi^{\omega}\}\}$, we solve SDP, described in (27), for each type by using an SDP solver, e.g., [20, 21], numerically.

• •

When we compute $S^{*}$ according to (28), we can compute the associated signaling strategies according to (30). Note that this step includes computation of eigen-decomposition of some matrices.

We re-emphasized that we provide an algorithm to compute robust signaling strategies globally even though the problem is highly nonlinear and non-convex. There is, however, still room to develop computationally more efficient approaches. And the solution concept proposed in the paper can be used as a benchmark to evaluate performance of such computationally efficient algorithms.

IV Noisy or Partial Measurements

In this section, we seek to obtain robust signaling strategies under noisy or partial (i.e., imperfect) measurements of the type (2). To this end, we turn the problem to the same structure with the case under perfect measurements based on a recent result from [28] and then invoke the results from the previous section.

There are several challenges in robust sensor design under imperfect measurements. For example, Proposition 1 does not hold anymore. To obtain a result similar to Proposition 1, a first attempt would be to focus on the covariance matrix of the posterior estimate of control-free measurements, i.e., $\mathrm{cov}\{\mathbb{E}\{\boldsymbol{y}_{k}^{o}|\boldsymbol{s}_{1:k}\}\}$, where $\boldsymbol{y}_{k}^{o}:=C\boldsymbol{x}_{k}^{o}+\boldsymbol{v}_{k}$, instead of $H_{k}=\mathrm{cov}\{\mathbb{E}\{\boldsymbol{x}_{k}^{o}|\boldsymbol{s}_{1:k}\}\}$. Quite contrary to the control-free state $\{\boldsymbol{x}_{k}^{o}\}$, however, control-free measurements $\{\boldsymbol{y}_{k}^{o}\}$ do not necessarily constitute a Markov process since in general $\mathbb{E}\{\boldsymbol{y}_{k}^{o}|\boldsymbol{y}_{1:k-1}^{o}\}\neq\mathbb{E}\{\boldsymbol{y}_{k}^{o}|\boldsymbol{y}_{k-1}^{o}\}$. Therefore, we focus on the augmented vector of measurements $\boldsymbol{y}_{1:k}^{o}\in\mathbb{R}^{nk}$.

Since the measurements are jointly Gaussian, we have

[TABLE]

This implies that the augmented measurements $\{\boldsymbol{y}_{1:k}\}_{k\geq 0}$ evolve according to

[TABLE]

where we define

[TABLE]

Note that neither $A_{k}\in\mathbb{R}^{nk\times n(k-1)}$ nor $\boldsymbol{e}_{k}\in\mathbb{R}^{nk}$ depend on signaling or control strategies. Furthermore, similar to (15), it can be shown that

[TABLE]

where we now have $\boldsymbol{s}_{k}^{o}=L_{k}^{\prime}\boldsymbol{y}_{1:k}^{o}+\boldsymbol{\vartheta}_{k}$.

Since $\boldsymbol{s}_{1:k}^{o}$ is $\sigma$-$\boldsymbol{y}_{1:k}$ measurable, $\boldsymbol{x}_{k}^{o}$ and $\boldsymbol{s}_{1:k}^{o}$ are independent of each other conditioned on $\boldsymbol{y}_{1:k}^{o}$. This implies that $\boldsymbol{x}_{k}^{o}$, $\boldsymbol{y}_{1:k}^{o}$, and $\boldsymbol{s}_{1:k}^{o}$ can be viewed as forming a Markov chain in the order $\boldsymbol{x}_{k}^{o}\rightarrow\boldsymbol{y}_{1:k}^{o}\rightarrow\boldsymbol{s}_{1:k}^{o}$. In that respect, [28, Lemma 1] shows that when $\{\boldsymbol{x}_{k}^{o},\boldsymbol{y}_{1:k}^{o},\boldsymbol{s}_{1:k}\}$ are jointly Gaussian and form a Markov chain in the order $\boldsymbol{x}_{k}^{o}\rightarrow\boldsymbol{y}_{1:k}^{o}\rightarrow\boldsymbol{s}_{1:k}^{o}$, there exists a linear relation between $\mathbb{E}\{\boldsymbol{x}_{k}^{o}|\boldsymbol{s}_{1:k}^{o}\}$ and $\mathbb{E}\{\boldsymbol{y}_{1:k}^{o}|\boldsymbol{s}_{1:k}^{o}\}$ irrespective of $\boldsymbol{s}_{1:k}^{o}$, and the relation is given by

[TABLE]

where $D_{k}\in\mathbb{R}^{m\times nk}$ is defined by

[TABLE]

Under imperfect measurements, counterpart of the covariance matrix $H_{k}=\mathrm{cov}\{\mathbb{E}\{\boldsymbol{x}_{k}^{o}|\boldsymbol{s}_{1:k}\}\}$ is the covariance matrix of the posterior estimate of control-free augmented measurements, denoted by

[TABLE]

Based on (30) and (31), we have

[TABLE]

Correspondingly, (17), i.e., the problem faced by $\mathcal{P}_{S}$, can be written as

[TABLE]

where we define101010We use the cyclic property of the trace operator. $W_{k}^{\omega}:=D_{k}^{\prime}\Xi^{\omega}D_{k}$, which can be viewed as the counterpart of $\Xi_{k}^{\omega}$ under imperfect measurements. We remark the resemblance between (17) and (32), where we have $Y_{k}$ instead of $H_{k}$ and $W_{k}^{\omega}$ instead of $\Xi_{k}^{\omega}$.

Recall that $\boldsymbol{y}_{k}\in\mathbb{R}^{n}$, which yields that $\boldsymbol{y}_{1:k}\in\mathbb{R}^{nk}$. Under the assumption that $\boldsymbol{s}_{k}\in\mathbb{R}^{nk}$ instead of $\boldsymbol{s}_{k}\in\mathbb{R}^{m}$ (so that $\mathcal{P}_{S}$ can disclose the auxiliary $\boldsymbol{y}_{1:k}$ perfectly), we would have transformed the problem under imperfect measurements into a problem under perfect measurements, and correspondingly we could have invoked the results from the previous section directly. The following lemma shows that results for the case where $\boldsymbol{s}_{k}\in\mathbb{R}^{nk}$ would also hold for the case where $\boldsymbol{s}_{k}\in\mathbb{R}^{m}$ even when $nk>m$.

Lemma 2**.**

Let us denote signaling strategies when $\boldsymbol{s}_{k}\in\mathbb{R}^{nk}$ by $\tilde{\eta}_{k}$ and the associated strategy space by $\tilde{\Upsilon}_{k}$. Then for any $\{\tilde{\eta}_{k}\in\tilde{\Upsilon}_{k}\}$, there exists a $\{\eta_{k}\in\Upsilon_{k}\}$ such that they both lead to the same control strategy and correspondingly the same cost for $\mathcal{P}_{S}$. And such a signaling strategy $\{\eta_{k}\in\Upsilon_{k}\}$ is given by

[TABLE]

Based on Lemma 2, the following corollary to Proposition 1 provides a tractable necessary and sufficient condition on $\{Y_{k}\}$.

Corollary 2** (A Necessary and Sufficient Condition Under Imperfect Measurements).**

For any signaling rule $\eta\in\Upsilon$, covariance matrix of posterior estimate of the control-free measurements, $\{Y_{k}=\mathrm{cov}\{\mathbb{E}\{\boldsymbol{y}_{1:k}^{o}|\boldsymbol{s}_{1:k}\}\}\}$, satisfies

[TABLE]

where $\Sigma_{k}^{y}:=\mathbb{E}\{\boldsymbol{y}_{1:k}^{o}(\boldsymbol{y}_{1:k}^{o})^{\prime}\}$.

Furthermore for any collection of positive semi-definite matrices $S_{1:\kappa}$ satisfying

[TABLE]

there exists a linear-plus-noise signaling strategy $\eta\in\Upsilon$ such that $S_{k}=Y_{k}$.

The following corollary provides a problem equivalent to (32).

Corollary 3**.**

The problem faced by $\mathcal{P}_{S}$, i.e., (32), is equivalent to

[TABLE]

subject to the following linear matrix inequalities:

[TABLE]

Furthermore, given as solution $\{S_{1:\kappa}\}$ for (36), we can compute an optimal signaling strategy for (32), as follows:

• •

Compute signaling strategies $\tilde{\eta}\in\tilde{\Upsilon}$ as if signal $\boldsymbol{s}_{k}$ can be $nk$ dimensional, i.e., $\boldsymbol{s}_{k}\in\mathbb{R}^{nk}$, according to (21), where we have $\Sigma_{k}^{y}$ instead of $\Sigma_{k}^{o}$, $A_{k}$ instead of $A$, and $\boldsymbol{y}_{1:k}$ instead of $\boldsymbol{x}_{k}$.111111We provide closed-form expressions for the auxiliary parameters $\Sigma_{k}^{y},A_{k}$, and $D_{k}$ in Appendix G for the reader’s reference.

• •

For computed $\tilde{\eta}\in\tilde{\Upsilon}$, compute associated signaling strategies $\eta\in\Upsilon$ according to (33).

We remark that under imperfect measurements optimal signaling strategies are not necessarily memoryless anymore.

Through a similar notational convention as in the previous section, we can write (36) as

[TABLE]

where we let $\bar{\Psi}\subset\mathbb{S}^{m\kappa(\kappa+1)/2}$ be the set corresponding to the constraints (37). Then, based on Corollary 3, the following corollary to Theorem 1 provides a computationally tractable way to compute robust sensor outputs under imperfect measurements.

Corollary 4** (Computing the Equilibrium Under Imperfect Measurements).**

The value of the Stackelberg equilibrium (38) is given by $\mu=\min_{\omega\in\Omega^{o}}\{\mu_{\omega}\}$, where

[TABLE]

Furthermore, let $\mu_{\omega^{*}}=\mu$ and

[TABLE]

Then, given $S^{*}\in\bar{\Psi}$, an associated linear-plus-noise signaling strategy $\eta\in\Upsilon$ can be computed according to Corollary 3.

V Illustrative Examples

In this section, we examine the performance of the proposed defense measure over various attack scenarios. As an illustrative example, we set length of the time horizon at $\kappa=10$, dimension of state $m=2$, and dimension of control input $r=2$. We consider the scenario where the system’s control objective is to track an exogenous process $\{\boldsymbol{z}_{k}\}$ evolving according to

[TABLE]

where $\boldsymbol{z}_{1}\sim\mathcal{N}(0,I_{2})$, and $\{\boldsymbol{w}_{k}^{z}\sim\mathcal{N}(0,I_{2})\}$ is a white Gaussian noise process, and they are independent of each other and every other parameter. Correspondingly, the system’s control objective can be written as

[TABLE]

We set $A=A^{z}=I_{2}$ and $B=\begin{bmatrix}1&1\\ 0&1\end{bmatrix}$ while $\Sigma_{1}=I_{2}$ and $\Sigma_{w}=I_{2}$. The sensor has access to the measurements:

[TABLE]

where $\{\boldsymbol{v}_{k}^{z}\}$ is a white Gaussian measurement noise independent of every other parameter.

Note that the control objective is not in the form of (5); however, we can transform it into the form of (5) by introducing the augmented state $\boldsymbol{\tilde{x}}_{k}:=\begin{bmatrix}\boldsymbol{x}_{k}^{\prime}&\boldsymbol{z}_{k}^{\prime}\end{bmatrix}^{\prime}$ evolving according to

[TABLE]

and augmented measurements are then given by

[TABLE]

Correspondingly (41) can be written as

[TABLE]

where

[TABLE]

As examples of attack scenarios, we consider a type set $\Omega=\{\omega_{a},\omega_{b},\omega_{c}\}$. Let us partition the underlying state $\boldsymbol{x}_{k}=\begin{bmatrix}\boldsymbol{x}_{k,1}&\boldsymbol{x}_{k,2}\end{bmatrix}^{\prime}$ and the exogenous process $\boldsymbol{z}_{k}=\begin{bmatrix}\boldsymbol{z}_{k,1}&\boldsymbol{z}_{k,2}\end{bmatrix}^{\prime}$, where $\boldsymbol{x}_{k,1}$ and $\boldsymbol{z}_{k,1}$ (or $\boldsymbol{x}_{k,2}$ and $\boldsymbol{z}_{k,2}$) correspond to the first (or the second) entries of the state and the exogenous process, respectively. We assume that type-$\omega_{a}$ attacker seeks to make $\{\boldsymbol{x}_{k,1}\}$ track $\{-\boldsymbol{z}_{k,1}\}$ instead of $\{\boldsymbol{z}_{k,1}\}$ whereas it is not interested in $\{\boldsymbol{x}_{k,2}\}$. Then its control objective can be written as

[TABLE]

Similarly type-$\omega_{b}$ attacker seeks to make $\{\boldsymbol{x}_{k,2}\}$ track $\{-\boldsymbol{z}_{k,2}\}$ instead of $\{\boldsymbol{z}_{k,2}\}$ whereas it is not interested in $\{\boldsymbol{x}_{k,1}\}$. Then its control objective can be written as

[TABLE]

On the other hand, type-$\omega_{c}$ attacker is interested in both entries and seeks to make the entire state $\{\boldsymbol{x}_{k}\}$ track $\{-\boldsymbol{z}_{k}\}$. Accordingly, its control objective can be written as

[TABLE]

We note that numerical simulations show that mixtures of types $\omega_{a}$, $\omega_{b}$ and $\omega_{c}$ can lead to larger costs for the system than any single type, including type-$\omega_{c}$. In the following, we examine the cost of $\mathcal{P}_{S}$ under perfect and imperfect measurements separately.

Under perfect measurements, the sensor has access to $\boldsymbol{\tilde{y}}_{k}=\boldsymbol{\tilde{x}}_{k}$. Note that perfect measurements provide the utmost freedom for $\mathcal{P}_{S}$ to shape the belief of the attacker. For any cost that $\mathcal{P}_{S}$ can attain under imperfect measurements, $\mathcal{P}_{S}$ can select a signaling strategy under perfect measurements to attain the same cost. Therefore, $\mathcal{P}_{S}$ attains the lowest possible cost under perfect measurements in the robust sensor design framework.

In Table I, we tabulate the cost to $\mathcal{P}_{S}$ for the scenarios where $i)$ there is no attack, i.e., type of $\mathcal{P}_{C}$ is $\omega_{o}$; $ii)$ there is an attack by type-$\omega_{a}$ attacker, i.e., type of $\mathcal{P}_{C}$ is $\omega_{a}$; $iii)$ there is an attack by type-$\omega_{b}$ attacker, i.e., type of $\mathcal{P}_{C}$ is $\omega_{b}$; and $iv)$ there is an attack by type-$\omega_{c}$ attacker, i.e., type of $\mathcal{P}_{C}$ is $\omega_{c}$. Cost to $\mathcal{P}_{S}$ varies depending on the type of $\mathcal{P}_{C}$ and how prepared $\mathcal{P}_{S}$ is while constructing the sensor outputs. In other words, $\mathcal{P}_{S}$ constructs the sensor outputs according to an extended type set.

For example, $\mathcal{P}_{S}$ would have constructed signaling strategies according to $\Omega^{o}=\{\omega_{o}\}$ if it views that there would not be any attack. Correspondingly, if there is no attack, then the cost would be $43.03$. However, if there is an attack by, e.g., type-$\omega_{b}$ attacker, then the cost would be $354.02$, which is significantly higher compared to $43.03$. Next, consider that $\mathcal{P}_{S}$ has constructed the sensor outputs according to $\Omega^{o}=\{\omega_{b}\}$. Then, the system would be prepared against an attack by type-$\omega_{b}$ attacker, and the cost would be $185.63$ when type-$\omega_{b}$ attacker attacks. This is significantly lower than the cost $354.02$ obtained when $\mathcal{P}_{S}$ constructs sensor outputs without any concern about possible attacks. However, now the cost to $\mathcal{P}_{S}$ is $119.09$ if there is no attack and the type of $\mathcal{P}_{C}$ is $\omega_{o}$. This is also higher than the cost $43.3$ obtained when $\mathcal{P}_{S}$ constructs sensor outputs without any concern about possible attacks. It is an uncertainty whether there will be an attack or not. Correspondingly, if $\mathcal{P}_{S}$ constructs the sensor outputs according to $\Omega^{o}=\{\omega_{o},\omega_{b}\}$, then the cost would be $115.29$ when there is no attack. It is lower than the cost $119.09$ obtained before. On the other hand, the cost would still be around $185.63$ when type-$\omega_{b}$ attacker attacks the system.

Even though $\mathcal{P}_{S}$ is prepared to an attack by type-$\omega_{b}$ attacker by constructing the sensor outputs according to $\Omega^{o}=\{\omega_{o},\omega_{b}\}$, there can be an attack by another type attacker, e.g., type-$\omega_{a}$. Then the cost would be $321.56$, which is significantly higher than the cost $185.63$ that would be obtained when type-$\omega_{b}$ attacker attacks. The system can decrease this cost by also considering the possibility of attacks by type-$\omega_{a}$ attacker while constructing the sensor outputs. For example, if $\mathcal{P}_{S}$ constructs the sensor outputs by taking into account types $\omega_{a}$, $\omega_{b}$, and $\omega_{c}$ attacks, then the cost would be at most $199.46$ if any of those types of attacks occurs and the cost would be $81.17$ if there is no attack. These examples show the importance of constructing sensor outputs in a robust way.

As an example for imperfect measurements, we take $\boldsymbol{y}_{k}=\boldsymbol{x}_{k,1}+\boldsymbol{x}_{k,2}+\boldsymbol{v}_{k}$, where $\boldsymbol{v}_{k}\sim\mathcal{N}(0,1)$, and $\boldsymbol{y}_{k}^{z}=\boldsymbol{z}_{k}+\boldsymbol{v}_{k}^{z}$, where $\{\boldsymbol{v}_{k}^{z}\sim\mathcal{N}(0,I_{2})\}$. In Table II, we tabulate the costs to $\mathcal{P}_{S}$ over various scenarios, similar to Table I. Table II shows that imperfect measurements lead to larger cost for the system when there is no attack, as to be expected. However, at certain scenarios, imperfect measurements can lead to better performance for the system. For example, when $\mathcal{P}_{S}$ constructs sensor outputs by considering that there would not be any attack, i.e., according to $\Omega^{o}=\{\omega_{o}\}$, and there is an attack by type-$\omega_{b}$ attacker, the cost would be $324.82$, which is lower than the cost $354.02$ obtained under perfect measurements. For this scenario, imperfect measurements end up obfuscating type-$\omega_{b}$ attacker and lead to lower cost. In that respect, robust sensor outputs can be viewed as optimal imperfect measurements that lead to minimum cost for the system. Furthermore, the cost to $\mathcal{P}_{S}$ increases under imperfect measurements over the scenarios where it is prepared. In Table II, we highlight those scenarios by shading their cells. A comparison of shaded cells of Tables I and II verifies the observation emphasized above, i.e., imperfect measurements limit $\mathcal{P}_{S}$’s ability to persuade $\mathcal{P}_{C}$.

VI Concluding remarks

In this paper, we have proposed and addressed persuasion-based robust sensor design as a security measure in control systems against attackers with unknown control objectives. By designing sensor outputs cautiously in advance, we have sought to shape attackers’ believes about the underlying state of the system in order to induce them to act/attack to the system in line with the system’s normal operation. We have modeled the problem formally under the solution concept of Stackelberg equilibrium where the defender/sensor is the leader. Non-strategic reaction of the follower/attacker implies that the defender faces an optimization problem while seeking to design robust sensor strategies. We have shown that the optimization problem is non-convex and highly nonlinear. To mitigate this issue, we have formulated a tractable problem equivalent to that problem and shown how to compute the associated signaling strategies. We have also extended the results to scenarios where there are imperfect measurements of the underlying state. Finally, we have examined the performance of the proposed framework across various attack scenarios.

Future directions of research on this topic include development of computationally efficient algorithms to compute optimal signaling rules and developing persuasion-based sensor design strategies for scenarios where attackers have partial information about the underlying state dynamics instead of full knowledge of it or have side information about the state instead of relying on sensor outputs only. Another interesting research direction would be its application on sensor placement or sensor selection.

Furthermore, even though we have motivated the framework by relating it to security, the framework could also address strategic information disclosure over multi-agent control networks where agents have different control objectives. Particularly, independent of how we motivate and set up the signaling problem (e.g., a security application or a multi-agent non-cooperative control network), the solution concept developed can be adopted in various settings in a straightforward way provided that

• •

Information of interest and all random variables/vectors are jointly Gaussian

• •

There is a single sender and possibly multiple receivers

• •

Optimal reaction of each receiver is linear in its posterior belief

• •

The sender’s objective depends on receivers’ reactions only through an arbitrary quadratic function of their posterior beliefs

In this paper, we have used this result to address uncertainties regarding attackers’ (or receivers’) objectives in the security of control systems over a finite horizon. However, the result could be adopted in several other scenarios as well, such as:

• •

over infinite horizon (as we did in [29])

• •

when there are additional tractable constraints on the covariance of the posterior belief (as we did in [28] for a power constraint over the signals when there is an additive Gaussian noise channel between the sender and the receiver)

Furthermore, the ability to turn signaling problems (which lead to highly nonlinear and non-convex optimization problems) into linear optimization problems (over the space of positive-semi definite matrices with linear matrix inequality constraints) facilitates analysis of problems over more complex settings, e.g., where

• •

There can be multiple controllers seeking to drive the same system

• •

There can be multiple senders that compete with each other to induce a controller to take certain actions

Appendix A Novelty Relative to Reference [9]

In Reference [9], we formulated an SDP equivalent to the original optimization problem in scenarios where there is no uncertainty on the attacker’s objective. Although it may seem to have a similar flavor, in [9] we used different technical tools and these tools cannot be adopted for the settings of this paper. Particularly, in [9] we exploited the fact that a solution of a linear optimization problem over a compact convex set lies at extreme points121212We say that a point in a convex set is an extreme point if it cannot be expressed as a convex combination of any other two points in the set. of the constraint set. Even though we were able to characterize the extreme points of the constraint set for the specific optimization problem in [9], characterization of extreme points is challenging in general, e.g., see [30]. Furthermore, in the settings of this paper, the associated optimization problem includes an inner maximization induced by the sensor’s robustness concern. Therefore the techniques developed in [9] cannot be used in this setting since the objective function is no longer linear in the optimization arguments due to the inner maximization. It is indeed a convex function since maximum of any family of linear/affine functions is a convex function [27]. Correspondingly, the solution does not necessarily lie at an extreme point of the constraint set. To be able to solve this optimization problem globally, in this paper we have shown that those linear matrix inequalities provide not only necessary but also sufficient conditions. This leads to a more comprehensive solution concept since it can be adopted in other sensor design settings in a straightforward way in order to obtain an equivalent tractable optimization problem, e.g., for the settings over imperfect measurements as we did here, infinite horizon as shown in [29] (based on the results of this paper), and several others.

Appendix B Proof of Lemma 1

Based on (16) and (12a), we obtain (17) through some algebra as detailed below. We focus on $i)$ part of the cost induced by the controller of the system, e.g., (9), and $ii)$ part of the cost induced by the attacker, e.g., (10), separately.

Part-$i)$ For notational brevity, let us first introduce the following matrices

[TABLE]

Then, the right hand side of (14) can be written in a compact form as

[TABLE]

in terms of the augmented vectors $\boldsymbol{u}=\begin{bmatrix}\boldsymbol{u}_{\kappa}^{\prime}&\cdots&\boldsymbol{u}_{1}^{\prime}\end{bmatrix}^{\prime}$ and $\boldsymbol{x}^{o}=\begin{bmatrix}(\boldsymbol{x}_{\kappa}^{o})^{\prime}&\cdots&(\boldsymbol{x}_{1}^{o})^{\prime}\end{bmatrix}^{\prime}$. Correspondingly, for type-$\omega$ $\mathcal{P}_{C}$, we obtain

[TABLE]

where $\boldsymbol{\hat{x}}^{o}:=\begin{bmatrix}\mathbb{E}\{\boldsymbol{x}_{\kappa}^{o}|\boldsymbol{s}_{1:\kappa}\}^{\prime}&\cdots&\mathbb{E}\{\boldsymbol{x}_{1}^{o}|\boldsymbol{s}_{1}\}^{\prime}\end{bmatrix}^{\prime}$ is the augmented vector of posteriors. Related to (9), this yields

[TABLE]

where we define $\Sigma^{o}:=\mathbb{E}\{\boldsymbol{x}^{o}(\boldsymbol{x}^{o})^{\prime}\}$,

[TABLE]

and $H:=\mathbb{E}\{\boldsymbol{\hat{x}}^{o}(\boldsymbol{\hat{x}}^{o})^{\prime}\}$, which follows since $\mathbb{E}\{\boldsymbol{x}^{o}(\boldsymbol{\hat{x}}^{o})^{\prime}\}=\mathbb{E}\{\boldsymbol{\hat{x}}^{o}(\boldsymbol{\hat{x}}^{o})^{\prime}\}$ due to the law of iterated expectations.

Part-$ii)$ The state $\boldsymbol{x}_{k}$ can be written in terms of control-free state $\boldsymbol{x}_{k}^{o}$ and control inputs $\boldsymbol{u}_{1:k}$ as follows:

[TABLE]

Let us define

[TABLE]

Then (42) and (45) yield that for $\omega\in\Omega$, we have

[TABLE]

where $\boldsymbol{w}:=\begin{bmatrix}\boldsymbol{w}_{\kappa}^{\prime}&\cdots&\boldsymbol{w}_{1}^{\prime}\end{bmatrix}^{\prime}$, $\bar{A}:=I_{\kappa}\otimes A$, $T^{\omega}:=Z(\Phi^{\omega})^{-1}K^{\omega}$, $\bar{Q}^{\omega_{o}}=I_{\kappa}\otimes Q^{\omega_{o}}$, and for all $\omega\in\Omega$ we define

[TABLE]

Note that (47) follows since $\{\boldsymbol{w}_{k}\}$ is a white noise and $T^{\omega}$ turns out to be an upper triangular (block) matrix.

Combining Parts $i)$ and $ii)$ together, we obtain that $\mathcal{P}_{S}$ faces the following problem:

[TABLE]

where $V^{\omega}$ and $\xi^{\omega}$ for $\omega\in\Omega^{o}$ are as described in (43), (44), (48), and (49). Based on the definition of $H$, it can be shown that

[TABLE]

Therefore, the corresponding $\Xi_{k}^{\omega}\in\mathbb{S}^{m}$ in (17) is defined by

[TABLE]

where $V_{k,l}^{\omega}\in\mathbb{R}^{m\times m}$ is an $m\times m$ block of $V^{\omega}$, with indexing from the right-bottom to the left-top.

Appendix C Proof of Proposition 1

The necessity condition has been shown in [9, Lemma 3].

In order to show the sufficiency of the condition, suppose that a collection of positive semi-definite matrices $S_{1:\kappa}$ satisfying (35) is given. Then $S_{1}\in\mathbb{S}^{m}$ satisfies

[TABLE]

Note that $\Sigma_{1}^{o}\succeq O$ can be singular. Therefore let $\Sigma_{1}^{o}=\bar{U}_{1}\begin{bmatrix}\bar{\Lambda}_{1}&O\\ O&O\end{bmatrix}\bar{U}_{1}^{\prime}$ be the eigen-decomposition such that $\bar{\Lambda}_{1}\in\mathbb{S}_{++}^{t_{1}}$ and $t_{1}:=\mathrm{rank}\{\Sigma_{1}^{o}\}$. When we multiply the terms in (51) from right by the unitary matrix $\bar{U}_{1}$ and from left by the transpose of the unitary matrix, i.e., $\bar{U}_{1}^{\prime}$, we obtain

[TABLE]

which implies that

[TABLE]

where we let $\bar{U}_{1}^{\prime}S_{1}\bar{U}_{1}=\begin{bmatrix}M_{1,1}&M_{1,2}\\ M_{2,1}&M_{2,2}\end{bmatrix}$ be the corresponding partitioning, e.g., $M_{1,1}\in\mathbb{S}^{t_{1}}$.

Since $\bar{U}_{1}^{\prime}S_{1}\bar{U}_{1}\succeq O$, we have $M_{2,2}\succeq O$ [31, Observation 7.1.2]. However, the bottom-right block of the positive semi-definite matrix (the whole term) on the left-hand-side of the inequality (52), i.e., $-M_{2,2}$, must also be a positive semi-definite matrix, which implies $O\succeq M_{2,2}$. Therefore we can conclude that $M_{2,2}=O$.

Next we invoke [32, Lemma 3] yielding $M_{1,2}=M_{2,1}^{\prime}=O$. Therefore (52) can be written as

[TABLE]

We define $T_{1}:=\bar{\Lambda}_{1}^{-1/2}M_{1,1}\bar{\Lambda}_{1}^{-1/2}$, let $T_{1}=U_{1}\Lambda_{1}U_{1}^{\prime}$ be its eigen-decomposition, and let $\lambda_{1,1},\ldots,\lambda_{1,t_{1}}$ be its eigenvalues. Then (53) yields that $I_{t_{1}}\succeq T_{1}\succeq O_{t_{1}}$ and correspondingly $I_{t_{1}}\succeq\Lambda_{1}\succeq O$, which implies that $\lambda_{1,i}\in[0,1]$ for all $i=1,\ldots,t_{1}$.

Since $M_{1,1}=\bar{\Lambda}_{1}^{1/2}T_{1}\bar{\Lambda}_{1}^{1/2}$, $M_{1,2}=M_{2,1}=O$, and $M_{2,2}=O$, $S_{1}$ can be written as

[TABLE]

Since $\bar{\Lambda}_{1}\in\mathbb{S}_{++}^{t_{1}}$ is not a singular matrix, (54) yields that there exists a bijective relation between $S_{1}\in\mathbb{S}^{m}$ and $T_{1}\in\mathbb{S}^{t_{1}}$, i.e., given $S_{1}$, we can compute $T_{1}$ and vice versa. Therefore, we can just focus on $T_{1}$ instead of $S_{1}$. To this end, consider a signaling strategy $\boldsymbol{s}_{1}=L_{1}^{\prime}\boldsymbol{x}_{1}+\boldsymbol{\vartheta}_{1}$, where $\boldsymbol{\vartheta}_{1}\sim\mathbb{N}(0,\Theta_{1})$. Then, the covariance matrix $H_{1}$ is given by

[TABLE]

Note that we can set $L_{1}\in\mathbb{R}^{m\times m}$ and $\Theta_{1}\in\mathbb{S}_{+}^{m}$ arbitrarily. Given the eigenvalues of $T_{1}$; it can be verified that if we set $L_{1}=\bar{U}_{1}\begin{bmatrix}\bar{\Lambda}^{-1/2}_{1}U_{1}\Lambda_{1}^{o}&O\\ O&O\end{bmatrix}$ and $\Theta_{1}\succeq O$ such that

[TABLE]

and the entries $\{\lambda_{1,i}^{o},\theta_{1,i}^{2}\}$ satisfy

[TABLE]

then we obtain $H_{1}=S_{1}$ exactly. Particularly, for such $L_{1,1}$ and $\Theta_{1}$, eigen-decomposition of $\Sigma_{1}^{o}$ yields that $L_{1}^{\prime}\Sigma_{1}L_{1}$ can be written as

[TABLE]

since unitary matrices satisfy $\bar{U}_{1}\bar{U}_{1}^{\prime}=\bar{U}_{1}^{\prime}\bar{U}_{1}=I$. On the other hand, $\Sigma_{1}^{o}L_{1}$ can be written as

[TABLE]

Therefore $\Sigma_{1}^{o}L_{1}(L_{1}^{\prime}\Sigma_{1}^{o}L_{1}+\Theta_{1})^{\dagger}L_{1}^{\prime}\Sigma_{1}^{o}$ can be written as

[TABLE]

which follows from (56). Recall that $T_{1}=U_{1}\Lambda_{1}U_{1}^{\prime}$. Therefore (57) is equivalent to (54), which verifies the claim. Note also that there always exist $\{\lambda_{1,i}^{o},\theta_{1,i}^{2}\}$ satisfying (56) since $\lambda_{1,i}\in[0,1]$ for all $i=1,\ldots,t_{1}$.

We have shown that given $S_{1}\in\mathbb{S}^{m}$ satisfying (51), we can select a signaling strategy such that $H_{1}=S_{1}$ exactly. Next, by following similar lines, we compute the associated signaling strategies for $k>1$ under the assumption that we have obtained them up to $k-1$.

Suppose that $H_{j}=S_{j}$ for $j<k$. Then, $S_{k}\in\mathbb{S}^{m}$ satisfies

[TABLE]

which is equivalent to

[TABLE]

Correspondingly, $\Sigma_{k}^{o}-AH_{k-1}A^{\prime}\succeq O$ can be singular. Let $\Sigma_{k}^{o}-AH_{k-1}A^{\prime}=\bar{U}_{k}\begin{bmatrix}\bar{\Lambda}_{k}&O\\ O&O\end{bmatrix}\bar{U}_{k}^{\prime}$ be the eigen-decomposition such that $\bar{\Lambda}_{k}\in\mathbb{S}_{++}^{t_{k}}$ and $t_{k}:=\mathrm{rank}\{\Sigma_{k}^{o}-AH_{k-1}A^{\prime}\}$. When we multiply the terms in (58) from right by the unitary matrix $\bar{U}_{k}$ and from left by the transpose of the unitary matrix, i.e., $\bar{U}_{k}^{\prime}$, we obtain

[TABLE]

Following the same reasons for the case $k=1$, [32, Lemma 3] yields that there exists a symmetric matrix $T_{k}\in\mathbb{S}^{t_{k}}$ such that

[TABLE]

and there exists a bijective relation between $T_{k}$ and $S_{k}-AH_{k-1}A^{\prime}$. Similarly, (59) and (60) yield that $I_{t_{k}}\succeq T_{k}\succeq O_{t_{k}}$, which implies that $T_{k}\in\mathbb{S}^{t_{k}}$ has eigenvalues in the closed interval $[0,1]$. Let $T_{k}=U_{k}\Lambda_{k}U_{k}^{\prime}$ be the eigen decomposition and $\lambda_{k,1},\ldots,\lambda_{k,t_{k}}\in[0,1]$ be the associated eigenvalues.

Furthermore, consider a signaling strategy $\boldsymbol{s}_{k}=L_{k}^{\prime}\boldsymbol{x}_{k}+\boldsymbol{\vartheta}_{k}$, where $\boldsymbol{\vartheta}_{k}\sim\mathbb{N}(0,\Theta_{k})$. Then, the covariance matrix $H_{k}$ is given by

[TABLE]

which follows since

[TABLE]

due to the independence of the jointly Gaussian $\boldsymbol{s}_{1:k-1}$ and $\boldsymbol{s}_{k}-\mathbb{E}\{\boldsymbol{s}_{k}|\boldsymbol{s}_{1:k-1}\}$. We can again set $L_{k}\in\mathbb{R}^{m\times m}$ and $\Theta_{k}\in\mathbb{S}_{+}^{m}$ arbitrarily. Given the eigenvalues of $T_{k}$, it can be verified that if we set $L_{k}=\bar{U}_{k}\begin{bmatrix}\bar{\Lambda}_{k}^{-1/2}U_{k}\Lambda_{k}^{o}&O\\ O&O\end{bmatrix}$ and $\Theta_{k}\succeq O$ such that

[TABLE]

and the entries $\{\lambda_{k,i}^{o},\theta_{k,i}^{2}\}$ satisfy

[TABLE]

then we would obtain $H_{k}=S_{k}$ exactly. Therefore, by induction, we conclude that for any $S_{1:\kappa}$ satisfying (35), there exists a certain signaling strategy $\eta\in\Upsilon$ such that $H_{k}=S_{k}$ for all $k=1,\ldots,\kappa$.

Appendix D Proof of Proposition 2

Since the objective function in (24) is continuous in the optimization arguments, and the constraint sets are decoupled and compact, the extreme value theorem and maximum theorem (showing the continuity of parametric maximization under certain conditions [33]) yields the existence of a solution to (24).

Appendix E Proof of Theorem 1

Based on the existence result in Proposition 2, suppose that $S^{*}$ solves (24) and $p^{*}$ is the maximizer of (25) for $S^{*}$. Since $p^{*}\in\Delta(\Omega^{o})$, there must be at least one type with positive weight. For example, suppose positive weight for the type $\omega\in\Omega^{o}$, i.e., $p_{\omega}>0$. This implies that

[TABLE]

since $\xi^{\omega}+\mathrm{Tr}\{S^{*}\Xi^{\omega}\}=\max_{\omega^{\prime}\in\Omega^{o}}\left(\xi^{\omega^{\prime}}+\mathrm{Tr}\{S^{*}\Xi^{\omega^{\prime}}\}\right)$ by (26). Furthermore, this also implies that

[TABLE]

since $\xi^{\omega}+\mathrm{Tr}\{S^{*}\Xi^{\omega}\}=\xi^{\omega^{\prime}}+\mathrm{Tr}\{S^{*}\Xi^{\omega^{\prime}}\}$ if $p_{\omega^{\prime}}^{*}>0$. These necessary conditions yield that (24) is equivalent to

[TABLE]

which is an SDP isolated from the distribution over the extended type set. Therefore, by searching over the extended type set $\Omega^{o}$, we can compute the minimum of (24), which is the minimum over $\Omega^{o}$. Once the minimum value is computed, $S^{*}$ can be computed according to the corresponding type, i.e., (28).

Appendix F Proof of Lemma 2

For a signaling strategy as described in (33), we would have

[TABLE]

where the last line follows since $\mathbb{E}\{\boldsymbol{x}_{l}^{o}|\tilde{\eta}_{1}(\boldsymbol{y}_{1}),\ldots,\tilde{\eta}_{l}(\boldsymbol{y}_{1:l})\}$, for $l\leq k$, is just a measurable function of $\{\tilde{\eta}_{1}(\boldsymbol{y}_{1}),\ldots,\tilde{\eta}_{l}(\boldsymbol{y}_{1:l})\}\}$ while $\mathbb{E}\{\boldsymbol{x}_{k}^{o}|\tilde{\eta}_{1}(\boldsymbol{y}_{1}),\ldots,\tilde{\eta}_{k}(\boldsymbol{y}_{1:k})\}$ is already conditioned on $\{\tilde{\eta}_{1}(\boldsymbol{y}_{1}),\ldots,\tilde{\eta}_{k}(\boldsymbol{y}_{1:k})\}$. This yields that they both would lead to the same posterior. Recall that the best reaction of $\mathcal{P}_{C}$ is linear function of $\mathbb{E}\{\boldsymbol{x}_{k}^{o}|\boldsymbol{s}_{1:k}\}$ as described in (16). Therefore they both would lead to the same control inputs almost surely and therefore the same cost for $\mathcal{P}_{S}$.

Appendix G Closed-Form Expressions for Auxiliary Parameters Under Imperfect Measurements

Note that $\Sigma_{k}^{y}\in\mathbb{S}^{nk}$, $A_{k}\in\mathbb{R}^{nk\times n(k-1)}$, and $D_{k}\in\mathbb{R}^{m\times nk}$ can be written as

[TABLE]