Secure Estimation under Causative Attacks

TL;DR

This paper develops a framework for secure parameter estimation under causative attacks that tamper with the statistical model, introducing detection and isolation mechanisms to improve robustness in inference algorithms.

Contribution

It introduces a novel secure estimation framework that incorporates attack detection and isolation, with closed-form decision rules and scalable algorithms for complex scenarios.

Findings

Established fundamental decision rules for secure estimation under attacks.

Developed a scalable estimation algorithm with optimality guarantees.

Applied the framework to sensor network parameter estimation.

Abstract

This paper considers the problem of secure parameter estimation when the estimation algorithm is prone to causative attacks. Causative attacks, in principle, target decision-making algorithms to alter their decisions by making them oblivious to specific attacks. Such attacks influence inference algorithms by tampering with the mechanism through which the algorithm is provided with the statistical model of the population about which an inferential decision is made. Causative attacks are viable, for instance, by contaminating the historical or training data, or by compromising an expert who provides the model. In the presence of causative attacks, the inference algorithms operate under a distorted statistical model for the population from which they collect data samples. This paper introduces specific notions of secure estimation and provides a framework under which secure estimation under causative attacks can be formulated. A central premise underlying the secure estimation framework is that forming secure estimates introduces a new dimension to the estimation objective, which pertains to detecting attacks and isolating the true model. Since detection and isolation decisions are imperfect, their inclusion induces an inherent coupling between the desired secure estimation objective and the auxiliary detection and isolation decisions that need to be formed in conjunction with the estimates. This paper establishes the fundamental interplay among the decisions involved and characterizes the general decision rules in closed-form for any desired estimation cost function. Furthermore, to circumvent the computational complexity associated with growing parameter dimension or attack complexity, a scalable estimation algorithm and its attendant optimality guarantees are provided. The theory developed is applied to secure parameter estimation in a sensor network.

Figures

Tables

Table 1. Table 1: Comparison of Estimation Performance from Optimal Decision Rules and Heuristic Approach

${\nu }_1^0$	${\nu }_1^1$	${\nu }_2^0$	${\nu }_2^1$	$\hat{q}$	$\alpha$	$\beta$	$q$
0.2	0.57	0.2	0.5261	1.48	0.0215	0.5134	1.126
0.25	0.4384	0.2	0.658	1.456	0.0216	0.6484	1.046
0.3	0.4467	0.15	0.6359	1.434	0.0482	0.5946	1.060
0.3	0.4467	0.3	0.4124	1.491	0.0762	0.4198	1.151
0.35	0.4021	0.25	0.4832	1.434	0.0528	0.4812	1.136
0.15	0.689	0.35	0.4124	1.44	0.0476	0.476	1.140

Equations

$$Y_r\sim f(\cdot |X),\text{mbox}withX\sim \pi .$$

$$\hat{X}(\mathbf{Y}):{\mathcal{Y}}^n\mapsto \mathcal{X}.$$

$$Y_r\sim f(\cdot |X),\text{mbox}withX\sim \pi ,\text{ for }r\in \{1,\dots n\}.$$

$$\begin{array}[]{ll}\vspace{0.1in}{\sf H}_{0}\ :&{\bf{{Y}}}\sim f({\bf{{Y}}}\mid X),\ \text{ with }X\sim\pi(X)\\ {\sf H}_{i}\ :&{\bf{{Y}}}\sim f_{i}({\bf{{Y}}}\mid X),\ \text{ with }X\sim\pi(X)\ ,\;\quad\mbox{for}\;\;\in\{1,\dots,T\}\end{array}\ .$$

$$\sum _{i=0}^T{\delta }_i(\mathbf{Y})=1.$$

$$\mathbb{P}(\mathsf{D}={\mathsf{H}}_j\mid \mathsf{T}={\mathsf{H}}_i)={\int }_{\mathbf{Y}}{\delta }_j(\mathbf{Y})f_i(\mathbf{Y})\mathrm{d}\mathbf{Y}.$$

$${\mathsf{P}}_{\mathsf{md}}(\mathbf{\delta })$$

$$=\frac{1}{\mathbb{P}(\mathsf{T}\ne {\mathsf{H}}_0)}\sum _{i=1}^T\mathbb{P}(\mathsf{D}\ne {\mathsf{H}}_i\mid \mathsf{T}={\mathsf{H}}_i)\mathbb{P}(\mathsf{T}={\mathsf{H}}_i)$$

$$=\sum _{i=1}^T\frac{{ϵ}_i}{1-{ϵ}_0}\cdot \mathbb{P}(\mathsf{D}\ne {\mathsf{H}}_i\mid \mathsf{T}={\mathsf{H}}_i).$$

$${\mathsf{P}}_{\mathsf{fa}}(\mathbf{\delta })$$

$${\mathsf{C}}_{\mathrm{p},i}(U(\mathbf{Y})\mid \mathbf{Y})\triangleq {\mathbb{E}}_i\left[\mathsf{C}(X,U(\mathbf{Y}))\mid \mathbf{Y}\right],\forall i\in \{0,\dots ,T\},$$

$$\hat{\mathbf{X}}(\mathbf{Y})\triangleq [{\hat{X}}_0(\mathbf{Y}),\dots ,{\hat{X}}_T(\mathbf{Y})].$$

$$J_i({\delta }_i,U_i(\mathbf{Y}))\triangleq {\mathbb{E}}_i[\mathsf{C}(X,U_i(\mathbf{Y}))\mid \mathsf{D}={\mathsf{H}}_i],\forall i\in \{0,\dots ,T\},$$

$$J(\mathbf{\delta },\mathbf{U})\triangleq \underset{i\in \{0,\dots ,T\}}{\max }{J}_i({\delta }_i,U_i(\mathbf{Y})),$$

$$J_0(V)=\mathbb{E}[\mathsf{C}(X,V(\mathbf{Y}))],$$

$$q(\mathbf{\delta },\mathbf{U},V)\triangleq \frac{J(\mathbf{\delta },\mathbf{U})}{J_0(V)}.$$

$$V^{\ast }\triangleq \arg \underset{V}{\min }{J}_0(V),\text{mbox}and{J}_0^{\ast }\triangleq \underset{V}{\min }{J}_0(V).$$

$$q\triangleq \underset{\mathbf{\delta },\mathbf{U}}{\min }q(\mathbf{\delta },\mathbf{U},V^{\ast }),\text{mbox}s.t.{\mathsf{P}}_{\mathsf{md}}(\mathbf{\delta })\le \beta .$$

$$\mathcal{Q}(\beta )$$

$$\mathcal{P}(\alpha ,\beta )$$

$${\mathsf{P}}_{\mathsf{md}}(\mathbf{\delta })$$

$${\mathsf{P}}_{\mathsf{fa}}(\mathbf{\delta })$$

$$\mathcal{P}(\alpha ,\beta )$$

$$\hat{\mathbf{X}}=\arg \underset{\mathbf{U}}{\min }J(\mathbf{\delta },\mathbf{U}).$$

$$\mathcal{P}(\alpha ,\beta )$$

$$U_i^{\ast }(\mathbf{Y})\triangleq \arg \underset{U_i(\mathbf{Y})}{\inf }{\mathsf{C}}_{\mathrm{p},i}(U_i(\mathbf{Y})\mid \mathbf{Y}).$$

$${\hat{X}}_i(\mathbf{Y})=U_i^{\ast }(\mathbf{Y}).$$

$$J(\mathbf{\delta },\hat{\mathbf{X}})=\underset{i}{\max }\left\{\frac{{\int }_{\mathbf{Y}}{\delta }_i(\mathbf{Y}){\mathsf{C}}_{\mathrm{p},i}^{\ast }(\mathbf{Y})f_i(\mathbf{Y})d\mathbf{Y}}{{\int }_{\mathbf{Y}}{\delta }_i(\mathbf{Y})f_i(\mathbf{Y})d\mathbf{Y}}\right\},$$

$${\mathsf{C}}_{\mathrm{p},i}^{\ast }(\mathbf{Y})\triangleq \underset{U_i(\mathbf{Y})}{\inf }{\mathsf{C}}_{\mathrm{p},i}(U_i(\mathbf{Y})\mid \mathbf{Y}).$$

$$f_i(\cdot |X)\sim \mathcal{N}({\theta }_i,X),\text{mbox}for{\theta }_i\in \mathbb{R},$$

$$X\sim {\mathcal{X}}^{-1}(\zeta ,\varphi ),$$

Full text

1 Introduction

1.1 Motivation

Statistical inference offers mechanisms for deducing the statistical properties of a population based on the data sampled from the population. Inference problems, broadly, focus on discerning the statistical model of the population or forming estimates about an unknown parameter that specifies the statistical model of the population. Anomaly detection, which has diverse applications in intrusion detection, fraud detection, fault detection, system health monitoring, and event detection, constitutes a major class of inference problems in which the objective is raising alarms when the data pattern (e.g., statistical model) deviates significantly from the expected patterns. Effective detection of anomalies in the data strongly hinges on the known rules for distinguishing normal and abnormal data segments. These rules, for instance, can be specified by an expert or by leveraging the historical data, depending on the context of the application. These rules, subsequently, can be used for designing inference algorithms for detecting the anomalies with specified guarantees on relevant figures of merit.

While anomaly detection, which in essence copes with the vulnerability of the sampled data to being contaminated or compromised, has been studied extensively (c.f. [1, 2, 3]), the vulnerability of the inference algorithms to being compromised is far less-investigated. The nature of security vulnerabilities that inference algorithms are exposed to is fundamentally distinct from that of data. Specifically, in the case of compromised sample data, the information of the decision algorithm about the model remains intact, while the data fed to the algorithm is anomalous, in which case the counter-measures consist of winnowing out the compromised samples and forming inferential rules that are robust against them. In contrast, attacks on the algorithms can be exerted by providing the algorithm with an incorrect statistical model for the data. This is viable by, for instance, contaminating the historical data or by confusing the expert that produces a model, which are critical for furnishing the true model for the statistical model of the data. Therefore, when the sampled data is compromised, an inference algorithm produces decisions based on an un-compromised known model for the data, while the data that it receives and processes are compromised. On the other hand, when the historical data or the expert are compromised, an inference algorithm functions based on an incorrect model for the data, in which case even un-compromised sampled data produces unreliable decisions.

The aforementioned security vulnerabilities for the inference algorithms can be capitalized on by adversaries in order to force an inference algorithm to deviate from its optimal structure and produce decisions in ways that serve an adversary’s purposes. Such attacks on decision algorithms are often referred to as causative attacks, through which an adversary aims to (i) make the inference algorithms oblivious to specific attacks, or (ii) degrade the performance of the inference algorithm in the presence of such an attack [4].

While the notion of secure decision-making in adjacent domains (e.g., machine learning and data mining) has been heavily investigated in recent years, the fundamental limits of secure statistical inference are not well-investigated, and all the limited existing studies remain rather ad-hoc. In this paper, we provide a framework for secure parameter estimation under the potential presence of causative attacks. We establish the fundamental tradeoffs involved in decision-making under causative attacks and characterize the optimal decision rules for securely estimating the parameters and concurrently detecting the presence of the attackers. Furthermore, we provide scalable algorithms for addressing settings in which the size or complexity of the attacks grow, and provide optimality guarantees on the performance of these algorithms. A summary of the content and the contributions are discussed in Section 1.2, and the relevant literature on secure statistical inference are reviewed in Section 1.3.

1.2 Overview and Contributions

To lay the context for discussing the problem investigated, consider the canonical parameter estimation problem in which we have a collection of probability distributions $\{P_{X}:X\in{\cal X}\}$ defined over a common measurable space. The objective is to estimate $X$, which lies in a known set ${\cal X}\subseteq\mathbb{R}^{p}$, from data samples ${\bf{{Y}}}\triangleq[Y_{1},\dots,Y_{n}]$, where the sample $Y_{r}$ is distributed according to $P_{X}$ and lies in a known set ${\cal Y}\subseteq\mathbb{R}^{m}$. We denote the probability density functions (pdfs) that the statistician assumes about the underlying distributions of $X$ and $Y_{r}$ by $\pi$ and $f(\cdot\;|\;X)$, respectively, i.e.,

[TABLE]

For convenience in the analysis, we will assume that the pdfs do not have any non-zero probability masses over lower-dimensional manifolds. The objective of the statistician is formalizing a reliable estimator

[TABLE]

Causative Attacks: In an adversarial environment, a malicious attacker might launch a causative attack to influence (degrade) the quality of $\hat{X}({\bf{{Y}}})$. The purpose of such an attack is to compromise the process that underlies acquiring the statistical models. We emphasize that such an attack is different from those that aim to compromise the data, e.g., false data injection attacks, which aim to distort the data samples ${\bf{{Y}}}$. Consequently, the effect of a causative attack is misleading the statistician about the true model $f(\cdot\;|\;X)$ that it assumes about the data. Such attacks are possible by compromising the historical (or training) data that is used for defining a model for the data. Depending on the specificity and the extent of a causative attack, e.g., the fraction of the historical or training data that is compromised, the true model $f(\cdot\;|\;X)$ can deviate to alternative forms, the space of which we denote by ${\cal F}$. The attack can affect the statistical distribution of any number of the $m$ coordinates of ${\bf{{Y}}}$. There are two major aspects to selecting ${\cal F}$ as viable model space.

• •

An attack is effective if the compromised model is sufficiently distinct from the model assumed by the statistician. Hence, even though in general ${\cal F}$ can be any representation of possible kernels $f(\cdot\;|\;X)$ mapping ${\cal Y}$ to $\mathbb{R}^{m}$, only a subset of such mappings suffices to describe the set of effective attacks.

• •

There exists a tradeoff between the complexity of the model space and its expressiveness. Specifically, if it is overly expressive, it can represent the possible compromised models with a more refined accuracy at the expense of more complex inferential rules.

The specifics of the attack model will be discussed in Section 2.

$(q,\beta)$-Security: The potential presence of an adversary introduces a new dimension to the estimation problem in (2). Specifically, on the one hand, the stochastic model of the data can be altered by an attack and detecting whether the data model is compromised, itself being an inference task, is never perfect. On the other hand, designing an optimal estimation rule strongly hinges on successfully isolating the true model. Hence, there exists an inherent coupling between the original estimation problem of interest and the introduced auxiliary problem (i.e., detecting the presence of an attacker and isolating the true model). Based on this observation, in an adversarial setting, there exists uncertainty about the true model, based on which the quality of the estimator is expected to degrade with respect to an attack-free setting. We are interested in establishing the fundamental interplay between the quality of discerning the true model and the degradation level in the estimation quality. To establish this interplay, we say that an estimator is $(q,\beta)$-secure if its estimation cost is weaker than that of the attack-free setting by a factor $q\in[1,+\infty)$, while missing at most $\beta\in(0,1]$ fraction of the attacks111Estimation costs and the associated estimation degradation factor $q$ will be defined in Section 2.3..

In this paper, we pursue three intertwined objectives. First, we characterize the fundamental tradeoffs between $q$ and $\beta$ and delineate a tradeoff curve. Secondly, we characterize the inference rules in closed-forms and provide a secure estimation algorithm that achieves the optimal tradeoffs for any desired point on the tradeoff curve. Finally, to circumvent the computational complexity as the the dimension of the data, $p$, grows, or the complexity of the attacks scales up (e.g., the number of coordinates compromised), we provide a scalable algorithm that has low computational complexity with guaranteed optimality in the asymptote of large data dimension $p$.

1.3 Related Studies

The problem of secure inference is studied primarily in the context of sensor networks. The study in [5], in particular, considers a two-sensor network in which one sensor is known to be secured, and one sensor is vulnerable to attacks. The objective is forming an estimate based on the mean-squared error criterion, for which a heuristic detection-driven estimator is designed. According to this design, first, a decision is formed about whether the unsecured sensor is attacked. If it is deemed to be attacked, then the estimator will rely only on the secured sensor, and otherwise, it uses the data from both sensors. Unlike in [5], we consider a model with arbitrary size, assume that all data coordinates are vulnerable to the attack, and characterize the optimal decision structure, which turns out to be different from being a detection-driven design studied in [5]. Through a case study, we will also show the rather significant improvement in the estimation quality when using the optimal rules, compared to the rules specified in [5].

The adversarial setting defined in this paper is also similar to the widely-studied Byzantine attack models in sensor networks, in which the data generated by the compromised sensors are modified arbitrarily by the adversaries in order to degrade or the inference quality. An overview of the impact of Byzantine attacks on inference quality in sensor networks and relevant mitigation strategies are discussed in [2]. Detection-driven estimation strategies (i.e., when attack detection precedes the estimation routine) when the effect of the Byzantine attacks are characterized by randomly flipping of the information bits generated by the sensors are discussed in [6, 7, 8, 9]. Furthermore, attack-resilient target localization strategies are investigated in [3] and [6], where it is assumed that the attacker adopts a fixed strategy for maximum disruption to the inference. In these studies, nevertheless, an attacker may deviate from the worst-case strategy of incurring the maximum damage in order to launch a less powerful but sustained attack, which may not be detected perfectly. Finally, strategies for isolating the compromised nodes in sensor networks are investigated in [10, 11, 12]. The emphasis of these studies is primarily focused on detecting attacks, or isolating the attacked sensors, which is different from the scope of our paper, which is focused on parameter estimation.

The problem of secure estimation in linear dynamical systems in the context of cyber-physical systems has been studied extensively in the recent years (c.f. [13, 14, 15, 16, 17, 18, 19]). The studies that are more relevant to the scope of this paper include [14], [18], and [19], which focus on the robust estimation of the states in dynamic systems. Specifically, a coding-theoretic interplay between the number of sensors compromised and the guarantees on perfect state recovery is characterized in [14], a Kalman filter-based approach for identifying the most reliable set of sensors to make an inference from is investigated in [18], and designing estimators that are robust against dynamical model uncertainty is studied in [19]. The degradation in estimation performance in a dynamical system consisting of a single sensor network is studied from the adversary’s perspective in [20], where the bounds on the degradation in estimation performance with degrees of stealthiness of the attacker are characterized.

All the aforementioned studies that involve secure estimation, irrespective of their focus or objective, conform in their design principle, which decouples the estimation decisions from all other decisions involved (e.g., attack detection or attacked sensor isolation), and produces either detection-driven estimators or estimation-driven detection routines. In the detection-driven estimation routines, an initial decision regarding the presence of an adversary (e.g., based on Neyman-Pearson theory) is followed by an optimal estimator based on the detection decision (e.g., Bayesian estimation). Such approaches implicitly assume that the detection decision has been perfect. Similarly, in an estimation-driven approach, the unknown parameter is first estimated, and then a detection decision is made (e.g., the generalized likelihood ratio test). Such approaches achieve optimality conditions only asymptotically, i.e., under the assumption of having an infinite number of measurements. The premise that decoupling such intertwined estimation and detection problems into independent estimation and detection routines is sub-optimal is well-investigated (e.g., in [21, 22, 23, 24]).

2 Data Model and Definitions

2.1 Attack Model

Our focus is on the canonical estimation problem in (2). The objective is to form an optimal estimate $\hat{X}({\bf{{Y}}})$ (under the general cost functions specified later) in the potential presence of a causative attack. Under the attack-free setting, the data is assumed to be generated according to the known distribution

[TABLE]

In an adversarial setting, an adversary, depending on its strength and preference, can launch an attack that can compromise the underlying process that the statistician uses for acquiring $f(\cdot\;|\;X)$. An attack will be carried out for the ultimate purpose of degrading the estimation quality of $X$. We assume that the adversary can corrupt the data model of up to $K\in\{1,\dots,m\}$ coordinates of ${\bf{{Y}}}$. Hence, for a given $K$, there exist $T=\sum_{i=1}^{K}{m\choose i}$ number of attack scenarios under which the compromised data models are distinct. Define ${\cal S}\triangleq\left\{S_{1},\dots,S_{T}\right\}$ as the set of all possible combinations of attack scenarios, where $S_{i}\subseteq\{1,\dots,m\}$ describes the set of coordinates the models of which are compromised under scenario $i\in\{1,\dots,T\}$.

Under the attack scenario $i\in\{1,\dots,T\}$, the joint distribution of $Y_{r}$ deviates from $f$ and changes to a model in the space ${\cal F}_{i}$. As discussed earlier, there exists a tradeoff between the expressiveness of this space and the complexity of the ensuing inferential rules. Specifically, a larger space ${\cal F}_{i}$ can distinguish different attack strategies with a more accurate resolution at the expense of high complexity in the analysis and the resulting decision rules. Also, the model can be effective if it encompasses sufficiently distinct models. Throughout the analysis of the paper, we assume that ${\cal F}_{i}\triangleq\{f_{i}(\cdot\;|\;X)\}$, i.e., ${\cal F}_{i}$ consists of one alternative distribution. This is primarily for the convenience in notations, and all the results presented can be generalized to any arbitrary space with countable elements. Based on this model, when the data models in the coordinates contained in $S_{i}$ are compromised, the joint distribution changes from $f(\cdot\;|\;X)$ to $f_{i}(\cdot\;|\;X)$.

Different attack scenarios might occur with different likelihoods, e.g., compromising one coordinate is easier than compromising two, and it might turn out to be more likely. To distinguish such likelihoods we adopt a Bayesian framework in which we define $\epsilon_{0}$ as the prior probability of having an attack-free scenario and define $\epsilon_{i}$ as the prior probability of the event that the attacker compromises the model under the coordinates specified by $S_{i}$. A block diagram of the attack model and the inferential goals to be characterized, which are discussed in the remainder of this section, is depicted in Fig. 1. Finally, we define the marginal pdf of the data at coordinate $l\in\{1,\dots,m\}$ under the attack-free setting and when the coordinate is compromised by $g_{l}^{0}$ and $g_{l}^{1}$, respectively.

2.2 Compound Decisions

The estimation objective constantly faces the uncertainty about whether an adversary exists. Furthermore, when one is deemed to exist, there is additional uncertainty pertaining to the number and the locations of the coordinates in which the data is being compromised. Hence, forming the estimate $\hat{X}({\bf{{Y}}})$ is inherently entwined with discerning the true model of the data. Decoupling the decisions for isolating the model and estimating the parameter under the isolated model does not generally render optimal performance. In fact, there exist extensive studies on formalizing and analyzing such compound decisions, which generally aim to decouple the inferential decisions. In [22], it is shown that generalized likelihood ratio test (GLRT) is not always optimal and necessary and sufficient conditions for its asymptotic optimality are provided. Moreover, note that GLRT utilizes maximum likelihood estimates of unknown parameters in its decision rule and thus, it is primarily focused on the detection performance. In [21], the problem of signal detection and estimation in noise is investigated under the Bayes criterion. In [23] and [24], non-asymptotic frameworks for optimal joint detection and estimation are developed. Specifically, in [23], a binary hypothesis testing problem is investigated in which one hypothesis is composite and consists of an unknown parameter to be estimated. In [24], the theory in [23] is extended to a composite binary hypothesis testing problem in which both hypotheses correspond to composite models.

2.3 Decision Cost Functions

2.3.1 Attack Detection Costs

The possibility of having multiple alternatives to the attack-free model renders the model detection problem as the following $(T+1)-$ composite hypothesis testing problem.

[TABLE]

where ${\sf H}_{0}$ is the hypothesis corresponding to the attack-free setting, and ${\sf H}_{i}$ is the hypothesis corresponding to an attack launched at the coordinates in $S_{i}\in{\cal S}$. For the convenience in notations, throughout the rest of the paper we denote the attack-free data model by $f_{0}(\cdot\;|\;X)$, i.e., $f_{0}(\cdot\;|\;X)=f(\cdot\;|\;X)$. To formalize relevant costs for the detection decisions, we define ${\sf D}\in\left\{{\sf H}_{0},\dots,{\sf H}_{T}\right\}$ as the decision on the hypothesis testing problem in (4), and ${\sf T}\in\left\{{\sf H}_{0},\dots,{\sf H}_{T}\right\}$ as the true hypothesis. We adopt a general randomized test $\boldsymbol{\delta}({\bf{{Y}}})\triangleq[\delta_{0}({\bf{{Y}}}),\dots,\delta_{T}({\bf{{Y}}})]$ for discerning the true hypothesis, where $\delta_{i}({\bf{{Y}}})\in[0,1]$ denotes the probability of deciding in favor of ${\sf H}_{i}$. Clearly,

[TABLE]

Hence, the likelihood of deciding in favor of ${\sf H}_{j}$ while the true model is ${\sf H}_{i}$ is given by

[TABLE]

We define ${\sf P}_{\sf md}$ as the aggregate probability of incorrectly identifying the true model under the presence of compromised coordinates, i.e.,

[TABLE]

Furthermore, we define ${\sf P}_{\sf fa}$ as the aggregate probability of erroneously declaring that a set of coordinates are compromised, while operating in an attack-free scenario. We have

[TABLE]

2.3.2 Secure Estimation Costs

In this subsection, we define two estimation cost functions for capturing the fidelity of the estimate $\hat{X}({\bf{{Y}}})$ we aim to form for $X$. For this purpose, we adopt a generic and non-negative cost function ${\sf C}(X,U({\bf{{Y}}}))$ to quantify the discrepancy between the ground truth $X$ and a generic estimator $U({\bf{{Y}}})$. Based on this, corresponding to each model ${\sf H}_{i}$ and given data ${\bf{{Y}}}$ we first define the average posterior cost function as

[TABLE]

where the conditional expectation is with respect to $X$ when the true model is ${\sf H}_{i}$. Besides this, due to having distinct data models under different attack models, we consider having possibly distinct estimators under different models. Driven by this, we consider the design for an estimate for $X$ under each model. We denote the estimate of $X$ under model ${\sf H}_{i}$ by $\hat{X}_{i}({\bf{{Y}}})$, and accordingly, we define

[TABLE]

Considering such distinct estimators, the estimation cost ${\sf C}(X,\hat{X}_{i}({\bf{{Y}}}))$ is relevant only if the decision is ${\sf H}_{i}$. Hence, for any generic estimator $U_{i}({\bf{{Y}}})$ of $X$ under model ${\sf H}_{i}$, we define the decision-specific average cost function as

[TABLE]

where the conditional expectation is with respect to $X$ and ${\bf{{Y}}}$. Accordingly, we define an aggregate average estimation cost according to

[TABLE]

where we have defined ${\bf{U}}\triangleq[U_{0}({\bf{{Y}}}),\dots,U_{T}({\bf{{Y}}})]$. Finally, corresponding to the attack-free scenario, in which the only possible data model is the assumed model $f$, corresponding to any generic estimator $V({\bf{{Y}}})$ we define the average estimation according to

[TABLE]

where the expectation is with respect to $X$ and ${\bf{{Y}}}$ under model $f$. It is noteworthy that $J_{0}$ defined in (14) is fundamentally different from $J({\boldsymbol{\delta}},{\bf{U}})$ defined in (13), since the former is the estimation cost when there is no alternative to $f$ (i.e., the attack-free scenario), while the latter is the estimation cost in an adversarial setting in which we have decided that the attacker has not compromised the data, which being a detection decision is never perfect and can be inaccurate with a non-zero probability. The role of $J_{0}(V)$ in our analysis is furnishing a baseline for the estimation quality in order to assess the impact of the potential presence of an adversary on the estimation quality.

3 Secure Parameter Estimation

In this section, we formalize the secure estimation problem. The core premise underlying the notion of secure estimation presented is that there exists an inherent interplay between the quality of estimating $X$ and the quality of isolating the true model governing the data. Specifically, perfect detection of an adversary’s attack model is impossible. At the same time, the estimation quality strongly relies on the successful isolation of the true data model. Lack of a perfect decision about the data model is expected to degrade the estimation quality compared to the attack-free scenario. To quantify such an interplay as well as the degradation in estimation quality with respect to the attack-free scenario, we provide the following definition.

Definition 1** (Estimation Degradation Factor)**

For a given estimator $V$ in the attack-free scenario, and a secure estimation procedure specified by the detection and estimation rules $({\boldsymbol{\delta}},{\bf{U}})$ in the adversarial scenario, we define the estimation degradation factor (EDF) as

[TABLE]

Based on this definition, next we define the performance region, which encompasses all the pairs of decision qualities $q({\boldsymbol{\delta}},{\bf{U}},V)$ and ${\sf P}_{\sf md}({\boldsymbol{\delta}})$ over the space of all possible decision rules $({\boldsymbol{\delta}},{\bf{U}},V)$.

Definition 2** (Performance Region)**

We define the performance region as the region of all simultaneously achievable estimation quality $q({\boldsymbol{\delta}},{\bf{U}},V)$ and detection performance ${\sf P}_{\sf md}({\boldsymbol{\delta}})$.

By leveraging the characteristics of the performance region, next we define the notion of $(q,\beta)$-security, which is instrumental in defining the secure estimation problem of interest. For this purpose, note that EDF normalizes the estimation cost in the adversarial setting by that of the attack-free scenario. The two estimation cost functions involved in $q({\boldsymbol{\delta}},{\bf{U}},V)$ can be computed independently, and as a result, determining their attendant decision rules can be carried out independently. For this purpose, we define $V^{*}$ as the optimal decision rule under the attack-free setting, and $J^{*}_{0}$ as the corresponding estimation cost, i.e.,

[TABLE]

Definition 3** ($(q,\beta)$-security)**

An estimation procedure specified by $({\boldsymbol{\delta}},{\bf{U}},V^{*})$ for the adversarial scenario is said to be $(q,\beta)$-secure if the decision rules $({\boldsymbol{\delta}},{\bf{U}})$ yield the minimal EDF among all the decision rules corresponding to which the average rate of missing the attacks does not exceed $\beta\in(0,1]$, i.e.,

[TABLE]

The performance region, and its boundary that specifies the interplay between $q$ and $\beta$ are illustrated in Fig. 2. Based on these definitions, we aim to characterize:

The region of all simultaneously achievable values of $q({\boldsymbol{\delta}},{\bf{U}},V^{*})$ and ${\sf P}_{\sf md}({\boldsymbol{\delta}})$, which is illustrated by the dashed region in Fig. 2. 2. 2.

The $(q,\beta)$-secure decision rules $({\boldsymbol{\delta}},{\bf{U}},V^{*})$ that solve (17), and specify the boundary of the performance region, which is illustrated by a solid line as the boundary of the performance region in Fig. 2.

By noting that $q({\boldsymbol{\delta}},{\bf{U}},V^{*})=\frac{J({\boldsymbol{\delta}},{\bf{U}})}{J^{*}_{0}}$, where $J^{*}_{0}$ is a constant, the performance region and the $(q,\beta)$-secure decision rules are found as the solutions to

[TABLE]

Solving ${\cal Q}(\beta)$ ensures that the likelihood of missing an attack is confined below $\beta$. However, it is insensitive to the rate of the false alarms that the decision rules can generate. In case the statistician wishes to also control the rate of false alarms, that is the rate of erroneously declaring an attack while there is no attack, we can further extend the notion of $(q,\beta)$-security as follows.

Definition 4

An estimation procedure is $(q,\alpha,\beta)$-secure if it is $(q,\beta)$-secure and the likelihood of false alarms does not exceed $\alpha\in(0,1]$.

The optimal decision rules that yield $(q,\alpha,\beta)$-secure decisions can be found as the solution to

[TABLE]

Remark 1

It can be easily verified that ${\cal Q}(\beta)={\cal P}(1,\beta)$.

Remark 2** (Feasibility)**

The probabilities ${\sf P}_{\sf md}({\boldsymbol{\delta}})$ and ${\sf P}_{\sf fa}({\boldsymbol{\delta}})$ cannot be made arbitrarily small simultaneously. Specifically, from the Neyman-Pearson theory [25], it can be readily verified that for any given $\alpha$, there exists a value $\beta^{\ast}(\alpha)$, which specifies the smallest feasible value for $\beta$.

We characterize the optimal solution to problems ${\cal P}(\alpha,\beta)$ and ${\cal Q}(\beta)$ in closed-forms in Section 4. Close scrutiny of the optimal decision rules indicates that the complexity of the rules grows exponentially with the dimension of $X$, i.e., $p$, and the number of coordinates that an adversary can compromise, i.e. $K$. To address the computational complexity, we address the scalability issue in Section 5. Specifically, we provide alternative low-complexity decision rules and show that despite their simple structure, they satisfy asymptotic optimality guarantees.

4 Secure Parameter Estimation: Optimal Decision Rules

In this section, we characterize an optimal solution to the more general problem ${\cal P}(\alpha,\beta)$, i.e., the estimators $\{\hat{X}_{i}({\bf{{Y}}}):i\in\left\{0,\dots,T\right\}\}$ and the detectors $\left\{\delta_{i}({\bf{{Y}}}):i\in\left\{0,\dots,T\right\}\right\}$. We will also specify how the decision rules simplify for characterizing the solution to the problem ${\cal Q}(\beta)={\cal P}(1,\beta)$. In order to proceed, we start by providing the expansion of the decision error probability terms ${\sf P}_{\sf md}({\boldsymbol{\delta}})$ and ${\sf P}_{\sf fa}({\boldsymbol{\delta}})$ in terms of the data models and decision rules. By noting (6) and leveraging (7), we have

[TABLE]

Similarly, by noting (6) and based on (9), we have

[TABLE]

By using the expansions in (20) and (21), the problem of interest in (19) can be equivalently cast as

[TABLE]

The roles of the estimators $\left\{U_{i}({\bf{{Y}}}):i\in\left\{0,\dots,T\right\}\right\}$ appear only in the utility function ${J}({\boldsymbol{\delta}},{\bf{U}})$. This allows for decoupling the optimization problem ${\cal P}(\alpha,\beta)$ into two sub-problems, as formalized in Theorem 1.

Theorem 1

The optimal secure estimators of $X$ under different models, i.e., $\hat{\bf{X}}=[\hat{X}_{0},\dots,\hat{X}_{T}]$ is the solution to

[TABLE]

Furthermore, the solution of ${\cal P}(\alpha,\beta)$, and subsequently the design of the attack detectors, can be found by equivalently solving

[TABLE]

This theorem establishes a property for the optimal estimator in (23). By leveraging this property, and also taking into account the decoupled structure of the problem ${\cal P}(\alpha,\beta)$ in (24), in the following theorem we provide optimal designs for the secure estimators. Interestingly, it is shown that the optimal estimator under each model can be specified by optimizing a relevant cost function defined exclusively for that model.

Theorem 2** ($(q,\alpha,\beta)$-secure Estimators)**

For the optimal secure estimators $\hat{\bf{X}}$ we have

The minimizer of the estimation cost $J_{i}(\delta_{i},U_{i}({\bf{{Y}}}))$, i.e., the estimation cost function under model ${\sf H}_{i}$, is given by

[TABLE] 2. 2.

The optimal estimator $\hat{\bf{X}}=[\hat{X}_{0},\dots,\hat{X}_{T}]$, specified in (23), is given by

[TABLE] 3. 3.

The cost function ${J}({\boldsymbol{\delta}},\hat{\bf{X}})$ is given by

[TABLE]

where we have defined

[TABLE]

*Proof: * See Appendix A.

For illustration purposes, in the next corollary, we provide the closed-forms of these decision rules when the distributions $\{f_{i}(\cdot\;|\;X):i\in\{0,\dots,T\}\}$ are all Gaussian.

Corollary 1** ($(q,\alpha,\beta)$-secure Estimators in Gaussian Models)**

When the data models are Gaussian such that

[TABLE]

where the mean values are distinct, and

[TABLE]

where ${\cal X}^{-1}(\zeta,\phi)$ denotes the inverse chi-squared distribution with parameters $\zeta$ and $\phi$, s.t., $\zeta+n>4$, for the optimal secure estimators $\hat{\bf{X}}$, we have:

The minimizer of the estimation cost $J(\delta_{i},U_{i}({\bf{{Y}}}))$, i.e., the estimation cost function under model ${\sf H}_{i}$, is given by

[TABLE] 2. 2.

The optimal estimator $\hat{\bf{X}}=[\hat{X}_{0},\dots,\hat{X}_{T}]$, specified in (23), is given by

[TABLE] 3. 3.

The cost function ${J}({\boldsymbol{\delta}},\hat{\bf{X}})$ is given by

[TABLE]

where we have

[TABLE]

Next, given the optimal estimators $\hat{{\bf{X}}}$, we characterize the optimal detection rules in the next theorem. The main observation is that even though we started by considering general randomized decision rules, these rules in their optimal forms reduce to deterministic ones. Furthermore, the decisions rules depend on the estimation costs that are computed based on the optimal estimation costs. These estimation costs make the decisions coupled. In order to proceed, we first show that problem ${\cal P}(\alpha,\beta)$ in (24) can be solved by leveraging the result of the following theorem, which specifies an auxiliary convex problem in a variational form.

Theorem 3

For any arbitrary $u\in\mathbb{R}_{+}$, we have ${\cal P}(\alpha,\beta)\leq u$ if and only if ${\cal R}(\alpha,\beta,u)\leq 0$, where we have defined

[TABLE]

Furthermore, ${\cal R}(\alpha,\beta,u)$ is convex, and ${\cal R}(\alpha,\beta,u)=0$ has a unique solution in $u$, which we denote by $u^{*}$.

*Proof: * See Appendix B.

The point $u^{*}$ has a pivotal role in specifying the optimal detection decision rules. We define the constants $\{\ell_{i}:i\in\left\{0,\dots,T+2\right\}\}$ as the dual variables in the Lagrange function associated with the convex problem ${\cal R}(\alpha,\beta,u^{*})$. Given $u^{*}$ and $\{\ell_{i}:i\in\left\{0,\dots,T+2\right\}\}$, the optimal detection rules can be characterized in closed-forms, as specified in the following theorem.

Theorem 4** ($(q,\alpha,\beta)$-secure Detection Rules)**

The optimal decision rules for isolating the compromised coordinates are given by

[TABLE]

where we have defined

[TABLE]

Constants $\{A_{0},\dots,A_{T}\}$ are specified by the data models, $u^{*}$, and its associated Langrangian multipliers $\{\ell_{i}:i\in\left\{0,\dots,T+2\right\}\}$. Specifically, we have

[TABLE]

and for $i\in\left\{1,\dots,T\right\}$ we have

[TABLE]

*Proof: * See Appendix C.

In the next corollary, we provide the closed-forms of these decision rules when the distributions $\{f_{i}(\cdot\;|\;X):i\in\{0,\dots,T\}\}$ are all Gaussian.

Corollary 2** ($(q,\alpha,\beta)$-secure Detection Rules in Gaussian Models)**

When the data models $\{f_{i}(\cdot\;|\;X):i\in\{0,\dots,T\}\}$ have the following Gaussian distributions

[TABLE]

where the mean values are distinct, and

[TABLE]

the optimal decision rules for isolating the compromised coordinates are given by

[TABLE]

where we have defined

[TABLE]

Constants $\{A_{0},\dots,A_{T}\}$ are specified by the data models, $u^{*}$, and its associated Langrangian multipliers $\{\ell_{i}:i\in\left\{0,\dots,T+2\right\}\}$. Specifically, we have

[TABLE]

and for $i\in\left\{1,\dots,T\right\}$ we have

[TABLE]

where ${{\sf C}}^{*}_{{\rm p},i}({\bf{{Y}}})$ is evaluated using (34) and we have

[TABLE]

By setting $T=1$, $n=1$, $\theta_{0}=0$, $\theta_{1}=2$, and selecting $\zeta=4,\phi=1$, Fig. 3 depicts the performance region and the associated $(q,\beta)$-security curve, which shows the tradeoff between the quality of the detection and the degradation in the estimation quality. It is noteworthy that this tradeoff is inherently due to the formulation of the secure estimation problem. Essentially, problem ${\cal P}(\alpha,\beta)$ as specified in (19), is designed to trade the quality of detection in favor of improving the estimation cost.

Based on all the decision rules specified in the section, and the detailed steps of specifying the parameters involved in characterizing the decision rules, we provide Algorithm 1, which summarizes all the steps for solving ${\cal P}(\alpha,\beta)$ for any feasible pair of $\alpha$ and $\beta$.

5 Scalable Secure Parameter Estimation

As the data dimension $m$ grows, the number of possible data models $T$ grows exponentially. This, subsequently, leads to an exponential growth in the complexity of forming the decision rules, e.g., the number of Lagrangian multipliers needed for characterizing the detection rules scales linearly in $T$. This can render Algorithm 1 computationally prohibitive. In order to circumvent the computational complexity, in this section, we develop a scalable approach to secure estimation, which exhibits optimality properties too. The core idea is to break down the problem into $m$ coordinate-level problems, treat them individually, and then aggregate the individual decisions. Specifically, the decisions involve a binary decision on whether the data is compromised. If the data is deemed to be compromised, then each coordinate is tested individually, and an estimate of $X$ is formed based on the data of that coordinate. Individual coordinate-level estimates are then tested for reliability, and combined to form an aggregate estimate for $X$. Since we need to perform only $m$ single-coordinate binary detection decisions followed by forming a coordinate-level estimation routine, the computational complexity scales only linearly in the number of coordinates $m$, as opposed to exponentially for forming the optimal decision rules.

5.1 Binary Attack Detection

In the first stage, we perform a binary test to detect whether data is compromised at all. This is carried out by solving a binary composite hypothesis testing problem given by

[TABLE]

where $\hat{{\sf H}}_{0}$ is the hypothesis corresponding to the attack-free setting, and $\hat{{\sf H}}_{1}$ signifies to the presence of an attack. Probability distribution $\hat{f}$ is a mixed distribution given by

[TABLE]

Similarly to the optimal approach of Section 4, to design the decision rules we define the randomized test $\boldsymbol{\hat{\delta}}\triangleq[\hat{\delta}_{0}({\bf{{Y}}}),\hat{\delta}_{1}({\bf{{Y}}})]$, in which $\hat{\delta}_{i}({\bf{{Y}}})\in\left[0,1\right]$ is the probability of deciding in favor of $\hat{\sf H}_{i}$, and we have $\delta_{0}({\bf{{Y}}})+\delta_{1}({\bf{{Y}}})=1$. Furthermore, define ${\sf D}_{\sf d}\in\{\hat{{\sf H}}_{0},\hat{{\sf H}}_{1}\}$ and ${\sf T}_{\sf d}\in\{\hat{{\sf H}}_{0},\hat{{\sf H}}_{1}\}$ as the decision and the true hypotheses. Hence, the false alarm rate is given by

[TABLE]

and the miss-detection rate is given by

[TABLE]

Since our aim is to maximize the true detection rate (or minimize (50)) in the first step, we design the decision rule $\boldsymbol{\hat{\delta}}$ using the Neyman Pearson approach. Following the Neyman-Pearson approach, as noted in Remark 2, we have the following decision rules for the binary detection decision  [25].

Theorem 5** (Attack Detection)**

The optimal attack detection rule that minimized the rate of missing the attacks subject to a constraint on the false alarms is given by

[TABLE]

*where the threshold $\gamma$ and the probability term $\varrho$ are chosen such that the false alarm constraint is satisfied with equality. *

5.2 Isolating Compromised Coordinates

The outcome of the binary detection rule in Section 5.1 is ruling whether there exists an attack in the data. If an attack is deemed to exists, in the second step, we carry out an isolation decision, the role of which is identifying the compromised coordinates. In this subsection, we start by analyzing the optimal isolation rules when an attack is deemed to exist in Section 5.2.1, and characterize the performance of the optimal decision rules. This analysis will serve as a baseline for comparing the performance of any alternative low-complexity isolation rule. Next, we provide an isolation rule in Section 5.2.2 that has low-complexity and can achieve the same performance asymptotically as that of the optimal decision rule.

5.2.1 Optimal Isolation Rule

If the attack detection rules specified by (51)-(52) determine that the data in one or more coordinates are compromised, in the next step we aim to identify the compromised coordinates. Isolating the set of compromised coordinates is equivalent to solving the following $T$-hypothesis testing problem.

[TABLE]

Define ${\sf D}_{\sf is}\in\{{\sf H}_{1},\dots,{\sf H}_{T}\}$ and ${\sf T}_{\sf is}\in\{{\sf H}_{1},\dots,{\sf H}_{T}\}$ as the decision formed and the true hypothesis, respectively. We define the randomized test $\boldsymbol{\hat{\delta}}_{1}({\bf{{Y}}})\triangleq[\hat{\delta}_{11}({\bf{{Y}}}),\dots,\hat{\delta}_{1T}({\bf{{Y}}})]$, in which $\hat{\delta}_{1i}({\bf{{Y}}})\in\left[0,1\right]$ is the probability of deciding in favor of $\hat{\sf H}_{i}$ given that we have already decided that part of the data is compromised, i.e., $\hat{\sf H}_{1}$ is the true hypothesis in (47). Accordingly, we define ${\sf P}_{\sf is}(\boldsymbol{\hat{\delta}}_{1})$ as the probability of making an erroneous decision on the problem in (53), given that the decision in the detection step was $\hat{{\sf H}}_{1}$. Therefore, ${\sf P}_{\sf is}(\boldsymbol{\hat{\delta}}_{1})$ is given by

[TABLE]

We also denote the error exponent of ${\sf P}_{\sf is}(\boldsymbol{\hat{\delta}}_{1})$ as the number of the samples $n$ grows according to

[TABLE]

In the next theorem, we characterize the optimal decision rule $(\boldsymbol{\hat{\delta}}_{1})$ and the associated error exponent. For this purpose, we denote the Chernoff information between two probability measures with probability density functions $g$ and $h$ by

[TABLE]

Theorem 6

The decision rule $\boldsymbol{\hat{\delta}}_{1}$ that minimizes ${\sf P}_{\sf is}(\boldsymbol{\hat{\delta}}_{1})$ is given by

[TABLE]

where

[TABLE]

*Proof: * See Appendix D.

Note that the optimal decision rules will have the same computational complexity as the optimal rules characterized in Theorem 4.

5.2.2 Asymptotically Optimal Isolation Rule

The optimal decision rule in Theorem 6 has similar drawbacks in terms of computational complexity in high dimensions as the optimal decision rules discussed in Section 4. In this subsection, we provide an alternative isolation rule for discerning the compromised coordinates at a lower computational complexity, and show that this rule is optimal in the asymptote of large number of samples, i.e., $n\rightarrow\infty$. We denote the alternative low-complexity decision rule by $\boldsymbol{\bar{\delta}_{1}}$ and start by providing lower and upper bounds on ${\sf P}_{\sf is}(\boldsymbol{\bar{\delta}_{1}})$, and show that these bounds exhibit the same error exponents. To specify a lower bound we first leverage the fact that ${\sf P}_{\sf is}(\boldsymbol{\hat{\delta}_{1}})\leq{\sf P}_{\sf is}(\boldsymbol{\bar{\delta}}_{1})$, which is true due to the optimality of ${\sf P}_{\sf is}(\boldsymbol{\hat{\delta}_{1}})$, and define ${\sf P}_{\sf is}(X,(\boldsymbol{\bar{\delta}}_{1}))$ as the value of ${\sf P}_{\sf is}(\boldsymbol{\bar{\delta}}_{1})$ when the unknown parameter $X$ is fully known (e.g., provided by a genie). Clearly, when $X$ is fully known, the error probability decreases, as part of the model uncertainty due to the unknown parameter is removed, and the decision problem reduces to a purely detection problem. To specify the upper bound on ${\sf P}_{\sf is}(\boldsymbol{\bar{\delta}}_{1})$, we define ${\sf P}_{\sf is}(X_{c},\boldsymbol{\bar{\delta}}_{1})$ as the value of ${\sf P}_{\sf is}(\boldsymbol{\bar{\delta}}_{1})$ when $X$ is assumed to take an arbitrary value $X_{c}\in\mathbb{R}^{p}$. Note that replacing $X$ by arbitrarily chosen $X_{c}$ induces sub-optimality compared to the optimal case in which optimal estimation is performed. The following lemma specifies the bounds on ${\sf P}_{\sf is}(\boldsymbol{\bar{\delta}}_{1})$.

Lemma 1

There exists some $X_{c}\in\mathbb{R}^{p}$, such that

[TABLE]

This lemma is instrumental in establishing the error exponent of the alternative low-complexity decision rule that we will provide. To proceed, corresponding to each coordinate $l\in\{1,\dots,m\}$ we define the likelihood ratio term

[TABLE]

where $g_{l}^{0}$ and $g_{l}^{1}$ denote the marginal pdfs of the data at coordinate $l\in\{1,\dots,m\}$ under the attack-free setting and when the coordinate is compromised, respectively. In the next theorem, we show that a decision rule based on calculating these likelihood ratio terms suffices to reach a decision that achieves the same error exponent as the optimal decision rule specified in (54).

Theorem 7

The isolation rule

[TABLE]

in which we have defined

[TABLE]

has the following error exponent

[TABLE]

which is equal to $\psi(\boldsymbol{\hat{\delta}}_{1})$, i.e., the error exponent of the optimal rule.

*Proof: * See Appendix E.

Therefore, the error probabilities corresponding to the optimal decision rule in Theorem 6 and the low-complexity alternative rule Theorem 7 decay at the same rate with the increasing number of samples $n$, rendering the decision rule in Theorem 7 asymptotically optimal. Clearly, the decision rule based on the marginal likelihood ratios significantly reduces the computational complexity for isolating the attacked coordinates. This reduction in computational complexity is viable at the expense of accuracy of the isolation when we have a small number of samples. In the next subsection, we discuss how the attack detection and coordinate isolation decisions characterized so far are leveraged for forming an estimate of $X$.

5.3 Coordinate-based Secure Estimation

The decision rules in Section 5.1 and Section 5.2 have already produced decisions about whether each individual coordinate is compromised. In the third stage of the decisions, we form one estimate of $X$ based on all the $n$ samples available at each coordinate. This leads to forming $m$ distinct estimates for $X$, one corresponding to each coordinate. Clearly, not all the estimates can be deemed reliable, especially when some of the coordinates are compromised. For this purpose, after forming the $m$ estimates we perform a reliability test, the purpose of which is discarding the unreliable estimates and retaining and aggregating the reliable ones. We provide relevant estimation cost functions in Section 5.3.1, and characterize the reliability test in Section 5.3.2.

5.3.1 Cost Functions

Based on the sequence of the data points at each coordinate, we form an estimate of $X$ corresponding to each coordinate, resulting in $m$ distinct estimates for $X$. Clearly, different coordinates produce estimates of $X$ with potentially different qualities. Motivated by the fact that the decisions formed in the previous steps are not perfect and a coordinate might yield an unreliable estimate for $X$, we consider performing a reliability test on the decision produced at each coordinate. For this purpose, at each coordinate $l$ and based on the data available at coordinate $l$, which we denote by ${\bf{{Y}}}_{l}$, we perform a binary test to decide whether the estimate based on the data from coordinate $l$ is reliable or unreliable, denoted by ${\sf H}^{\sf r}_{l}$ and ${\sf H}^{\sf u}_{l}$, respectively.

We define ${\sf D}^{\sf r}_{l}\in\{{\sf H}^{\sf r}_{l},{\sf H}^{\sf u}_{l}\}$ as the decision formed about the reliability of the estimate, and let the randomized test $\bar{\boldsymbol{\delta}}_{l}({\bf{{Y}}}_{l})=[\bar{\delta}_{l}^{\sf r}({\bf{{Y}}}_{l}),\bar{\delta}^{\sf u}_{l}({\bf{{Y}}}_{l})]$ be the decision rule to decide upon the reliability of the estimate from coordinate $l$, where $\bar{\delta}_{l}^{\sf r}({\bf{{Y}}}_{l})$ is the probability of deciding in favor of ${\sf H}^{\sf r}_{l}$ and $\bar{\delta}_{l}^{\sf u}({\bf{{Y}}}_{l})$ is the probability of deciding in favor of ${\sf H}^{\sf u}_{l}$. Furthermore, we define $\boldsymbol{\delta}_{l}({\bf{{Y}}})\triangleq[\delta^{0}_{l}({\bf{{Y}}}_{l}),\delta^{1}_{l}({\bf{{Y}}}_{l})]$, where $\delta^{1}_{l}({\bf{{Y}}}_{l})$ denotes the probability that coordinate $l$ is being compromised based on data ${\bf{{Y}}}$ and the decision rules in Section 5.1 and Section 5.2, and subsequently, $\delta^{0}_{l}({\bf{{Y}}}_{l})=1-\delta^{1}_{l}({\bf{{Y}}}_{l})$. Hence, the likelihood of forming a reliable estimate at coordinate $l$ given that we have decided that coordinate $l$ is not compromised is given by

[TABLE]

Similarly, the likelihood of forming a reliable estimate at coordinate $l$ given that we have decided that coordinate $l$ is compromised is given by

[TABLE]

Furthermore, consider $U^{0}_{l}$ and $U^{1}_{l}$ as the estimates of $X$ at coordinate $l$ when we have decided that the coordinate is attack-free and compromised, respectively. Hence, the cost associated with $U_{l}^{j}$ when the decision of the reliability test is ${\sf H}^{\sf r}_{l}$ is defined as

[TABLE]

where ${\sf C}^{j}_{l}(U_{l}^{j}\mid{\bf{{Y}}}_{l})$ is the posterior estimation cost at coordinate $l$. We define the optimal average estimation cost as

[TABLE]

and the optimal coordinate-level estimators as

[TABLE]

5.3.2 Reliability Decision Rules

For the probabilities of forming a reliable estimate at coordinate $l$ defined in (65) and (66) we have

[TABLE]

where $(1-\rho_{l}^{0})$ and $(1-\rho_{l}^{0})$ are the Type-I and Type-II probabilities of mis-classifying the status of coordinate $l$. Therefore, the probability of forming a reliable estimate, when coordinate $l$ is deemed to be compromised or attack-free is upper bounded by $\rho^{1}_{l}$ and $\rho_{l}^{0}$, respectively. This implies that only a fraction of the decisions on the true model of data at coordinate $l$ will provide reliable estimates. We wish to have decision rules that control the fraction of the reliable estimates to be beyond a pre-specified level. Obviously, these desired levels should also be within the feasible range. For this purpose, we select $\nu_{l}^{j}\in[0,\rho_{l}^{j}]$, and impose a lower bound on the fraction of the reliable estimates that the reliability test retains according to

[TABLE]

Based on the cost functions and decision constraints, the decision rules $\boldsymbol{\bar{\delta}}_{l}$ and the estimators $U_{l}^{j}$ are determined by solving $m$ problems in parallel, where the problem to be solved corresponding to coordinate $l$ is given by

[TABLE]

Similarly to the discussion in Section 4, since the effect of the estimators $U_{l}^{j}$ appears only in the cost function ${J}_{l}^{j}(\delta_{l}^{j},\bar{\delta}_{l}^{\sf r},U_{l}^{j})$, the optimization problem in (76) can be decoupled into two subproblems as formalized in the following theorem.

Theorem 8

For given $\nu_{l}^{0}$ and $\nu_{l}^{1}$, the optimal decision rules for the problems ${\cal S}(\nu_{l}^{0})$ and ${\cal S}(\nu_{l}^{1})$ are given by

[TABLE]

where $\gamma_{l}^{1}$ and $\gamma_{l}^{0}$ are selected such that the constraints in (76) are satisfied with equality. The optimal estimate that solve ${\cal S}(\nu_{l}^{0})$ and ${\cal S}(\nu_{l}^{1})$ are the estimators $\hat{X}_{l}^{0}({\bf{{Y}}}_{l})$ and $\hat{X}_{l}^{1}({\bf{{Y}}}_{l})$, respectively, defined in (71).

*Proof: * See Appendix F.

For given data ${\bf{{Y}}}$, performing the reliability test in Theorem 8 retains the estimates deemed reliable. These estimates, subsequently, can be aggregated to form a single estimate for $X$. The estimation approach, which consists of global binary attack detection, followed by coordinate-level isolation of compromised coordinates and local estimates, and completed by aggregating the reliable coordinate-level estimates, renders a scalable approach to the secure estimation of interest. This approach is significantly less complex as compared to the optimal estimation rules characterized in Theorem 2. Finally, we remark that aggregating the local estimates is a well-investigated problem, to which there exists a broad range of solutions with varying degrees of optimality under various settings. Representative approaches to such aggregation include the studies in [26, 27, 28, 29]. A summary of the steps involved in the scalable approach is provided in Algorithm 2.

6 Case Studies: Secure Estimation in Sensor Networks

We use the example of a sensor network consisting of two sensors and a fusion center (FC) to evaluate the estimation frameworks presented in this paper. Each sensor is collecting a stream of data. Sensor $i\in\{1,2\}$ collects $n$ measurements, denoted by ${\bf{{Y}}}_{i}=[Y^{i}_{1},\dots,Y^{i}_{n}]$, where each sample $Y^{i}_{j}\in\mathbb{R}$ in an attack-free environment is related to $X$ according to

[TABLE]

where $h^{i}$ models the channel connecting sensor $i$ to the FC and $N_{j}^{i}$ accounts for the additive channel noise. Different noise terms are assumed to be independent and identically distributed (i.i.d.) generated according to a known distribution. We will consider two adversarial scenarios that impact the data model in (79) and evaluate the optimal performance, as well as the application of the asymptotically optimal scalable algorithm when the number of the samples $n$ tends to infinity.

6.1 Case 1: One Sensor Vulnerable to Causative Attacks

We start by considering an adversarial setting in which the data model of the measurements from only one sensor (sensor 1) are vulnerable to a causative attack, while the other sensor (sensor 2) remains attack-free. Under such a setting, we have only one attack scenario, i.e., $T=1$ and $S_{1}=\{1\}$. Accordingly, we have $\epsilon_{0}+\epsilon_{1}=1$. Under the attack-free scenario, we assume that the noise terms $N_{j}^{i}$ are i.i.d. and distributed according to ${\cal N}(0,\sigma^{2}_{n})$, i.e.,

[TABLE]

When data from sensor $1$ is compromised, the actual conditional distribution of $Y_{j}^{1}|X$ is distinct from the above distribution assumed by the statistician. The inference objective under such a setting, in principle, becomes similar to the adversarial setting of [5], which focuses on data injection. Hence, in order to be able to compare the performance of the optimal framework with that of [5], we assume that the conditional distribution of $Y_{j}^{1}|X$ when sensor 1 is under a causative attack is ${\cal N}(h^{i}X,\sigma^{2}_{n})\ast{\sf Unif}[a,b]$, where $a,b\in\mathbb{R}$ are fixed constants and $\ast$ denotes convolution. Therefore, the composite hypothesis test for estimating $X$ and discerning the model in (4) simplifies to the following binary test with the prior probabilities $\epsilon_{0}$ and $\epsilon_{1}$, in which we have defined ${\bf{{Y}}}=[{\bf{{Y}}}_{1},{\bf{{Y}}}_{2}]$.

[TABLE]

Figure 4 depicts the variations of the estimation quality, captured by $q$, versus the tolerable miss-detection rate $\beta$, where it is observed that the estimation quality improves monotonically as $\beta$ increases, and it reaches its maximum quality when $\beta=1$. This observation is in line with what is expected analytically from the formulation of the secure parameter estimation problems in (18) and (19).

A similar setting is studied in [5], where the attack is induced additively into the data of sensor 1 and can be any real number. This setting can be studied in the context of causative attacks where the attacker’s mode of compromising the data is adding a disturbance which has uniform distribution. Figure 4 also compares the estimation quality of the methodology developed in this paper, with that obtained by applying the methodology of [5], which characterizes a single point in the $(q,\beta)$ plane. Specifically, in [5], an estimator is designed to obtain the most robust estimate by exploring the dependence of the estimation quality on the false alarm probability, using which an optimal false alarm probability $\alpha^{\ast}$ is obtained, which in turn, fixes the miss-detection error probability, and does not provide the flexibility to change the miss-detection rate $\beta$.

The results presented in Fig. 4 correspond to $\sigma=3$, $\sigma_{n}=1$, $h^{1}=1$, $h^{2}=4$, $a=-40,b=40$. The upper bound on ${\sf P}_{\sf fa}$ is set to $\alpha^{\ast}=0.1$, where $\alpha^{\ast}$ is obtained using the methodology in [5].

6.2 Case 2: Both Sensors Vulnerable to Causative Attacks

We again consider the same model for $X$, and in this setting, we assume that data from both the sensors are vulnerable to being compromised. We assume that the attacker can compromise the data of at most one sensor at any instant. Under such a setting, we have $T=2$, $S_{1}=\{1\}$, and $S_{2}=\{2\}$. Therefore, under the adversarial setting, the sensor measurements follow the following composite hypothesis model

[TABLE]

where ${\sf H}_{0}$ corresponds to the attack-free setting, and hypothesis ${\sf H}_{i}$ corresponds to the data of sensor $i$ being compromised. Motivating by the fact that the sensor with the higher gain $h^{i}$ is expected to generate a better estimate, we explore a scenario in which the sensor with the higher gain is more likely to be attacked. Hence, we select the parameters $h^{1}=1$, and $h^{2}=2$, and set the probabilities $(\epsilon_{0},\epsilon_{1},\epsilon_{2})=(0.2,0.2,0.6)$. We set the distribution of $X$ to ${\sf Unif}[-2,2]$. We assume that $Y_{j}^{i},$ for $i\in\{1,2\}$, given $X$, is distributed according to ${\cal N}(h^{i}X,1)$ in the attack-free setting. When sensor $i$ is compromised, we assume that $Y_{j}^{i},$ for $i\in\{1,2\}$, given $X$, follows the distribution ${\cal N}(h^{i}X,5)$.

Figure 5 depicts the performance region defined in Fig. 2, which specifies the variations of $q$ versus $\beta$ for three different values of $\alpha$. The region spanned by the plots between $q$ and $\beta$ for different values of $\alpha$ is the feasible region of operation for the secure estimation methodology developed in this paper. This provides the FC with the flexibility to adjust the emphasis on each of the estimation or detection decisions. As expected, the estimation quality improves monotonically as $\alpha$ and $\beta$ increase.

6.3 Case 3: Evaluation of Scalable Secure Parameter Estimation Framework

Before comparing the estimation performance from the scalable approach developed in Section 5 and the optimal approach, we illustrate the asymptotic optimality of the decision rule proposed in Theorem 7. Consider a 2 sensor network, where the measurements of at least one sensor are compromised at any instant. The measurements of the sensors follow a similar model as described in (79). Assuming that the parameter $X$ follows the distribution ${\cal N}(0,\sigma^{2})$, $Y_{j}^{i}$ given $X$ is distributed according to ${\cal N}(h^{i}X,\sigma_{1}^{2})$ under the attack-free scenario and according to ${\cal N}(h^{i}X,\sigma_{2}^{2})$ when sensor $i$ is compromised, we illustrate the variations of $-\log({\sf P}_{\sf is}(\boldsymbol{\bar{\delta}}_{1}))$ versus the number of observations at each sensor in Fig. 6. For the results in Fig. 6, we set $h^{1}=1,h^{2}=4$, $\sigma^{2}=4$, $\sigma_{1}^{2}=2$ and $\sigma_{2}^{2}=6$ . The error probabilities corresponding to both decision rules decay exponentially at the same rate with the increase in number of observations at each sensor.

In order to compare the performance of the scalable secure estimation framework with that of the optimal framework, we choose $\nu_{l}^{0}$ and $\nu_{l}^{1}$, for $l\in\{1,2\}$, according to the steps described in Algorithm 2. We aggregate the reliable local estimates to form an optimal linear estimate at the FC using the fusion strategy for sensor networks with star topology in [28]. The decision on the reliability of the estimate is formed using the decision rules in Theorem 8, following which the local linear estimates from the sensors deemed to provide reliable estimates are aggregated at the FC. To compare with the estimation degradation factor $q$ for an optimal framework, we evaluate the estimation degradation factor for the scalable secure estimation framework and denote it by $\hat{q}$. The estimation performances for the scalable framework and the optimal decision rules are compared in the following table:

For the results presented in Table 1, we have set $h^{1}=h^{2}=1$. We assume that the parameter $X$ is distributed according to ${\cal N}(0,3)$, $N_{j}^{i}$ is distributed according to ${\cal N}(0,1)$ under the attack-free scenario, and according to ${\cal N}(0,1)\ast{\sf Unif}[-10,10]$ when data from sensor $i$ is compromised. As expected, the optimal decision rules result in superior estimation quality as compared to that obtained from the scalable framework, i.e., $\hat{q}>q$.

7 Conclusion

We have formalized and analyzed the problem of secure parameter estimation problem under the potential presence of causative attacks on the estimation algorithm. Under causative attacks, the information of the estimation algorithm about the statistical model of the sampled data is compromised. This leads the estimation algorithm exhibit degraded performance compared to the attack-free setting. We have provided closed-form optimal decision rules that ensure the best estimation quality (minimum estimation cost) while controlling the error in detecting the attacks and isolating the true model of the data. We have shown that the design of optimal estimators is intertwined with the detection rules for deciding upon the true model of the data. Based on this, we have designed the optimal decision rules, which combine both estimation performance and detection power. Based on this vision, the decision-maker can place any desired emphasis on the estimation and detection routines involved. to study the trade-off between the two. We have also provided case studies by applying the theory developed to sensors networks, where sensors face security vulnerabilities. Finally, to circumvent the computational complexity associated with growing the data dimension or attack complexity, we have provided a low-complexity secure estimation algorithm that is optimal in the asymptote of high data dimension.

Appendix A Proof of Theorem 2

From (12) we have

[TABLE]

Using the definition of ${\sf C}_{{\rm p},i}(U_{i}({\bf{{Y}}})\mid{\bf{{Y}}})$ from (10), a lower bound on ${J}_{i}({\delta_{i},U_{i}({\bf{{Y}}})})$ is given by

[TABLE]

which implies that

[TABLE]

Based on the definition of $\hat{X}_{i}({\bf{{Y}}})$ provided in (25), this lower bound is clearly achieved when the estimator $U_{i}({\bf{{Y}}})$ is chosen to be

[TABLE]

which proves that the estimator characterized in (25) is an optimal estimator that minimizes the cost ${J}_{i}(\delta_{i},U_{i})$. The corresponding minimum average estimation cost is

[TABLE]

Next, we prove that

[TABLE]

Recall from (13), the overall estimation cost ${J}(\boldsymbol{\delta},{{\bf{U}}})$ is defined as

[TABLE]

Define ${\cal C}(\boldsymbol{\Omega,\delta},{\bf{U}})$ as a convex function of ${J}_{i}(\delta_{i},U_{i}),i\in\left\{0,\dots,T\right\}$, given by

[TABLE]

where $\boldsymbol{\Omega}=[\Omega_{0},\dots,\Omega_{T}]$, and $\Omega_{i}$ satisfy

[TABLE]

We can represent ${J}(\boldsymbol{\delta},{\bf{U}})$ as a function of ${\cal C}(\boldsymbol{\Omega},\boldsymbol{\delta},{\bf{U}})$ in the following form

[TABLE]

Let $\boldsymbol{\Omega^{\ast}}=\{\Omega_{j}^{\ast}:j=0,\dots,T\}$ be defined as

[TABLE]

where $\Omega_{j}^{\ast}=1$ if

[TABLE]

From (88) and (89), we observe that

[TABLE]

Also, at the same time, we have

[TABLE]

which implies that

[TABLE]

From (A) and (97), it is easily concluded that

[TABLE]

which completes the proof for (90). Using the results in (90) and (89), the cost function ${{J}}(\boldsymbol{\delta},\hat{{\bf{X}}})$ is given by

[TABLE]

Appendix B Proof of Theorem 3

The function ${J}_{i}(\delta_{i},U_{i})$ is a quasi-convex function. Since the weighted maximum function preserves the quasi-convexity, it is concluded that ${{J}}_{i}(\delta_{i},\hat{X}_{i})$ is a quasi-convex function from its definition in (27). Therefore, we can find the solution by solving an equivalent feasibility problem given below [30]. Specifically, for $u\in\mathbb{R_{+}}$, it is easily observed that

[TABLE]

Hence, the feasibility problem equivalent to (24) is given by

[TABLE]

The above problem is feasible if ${\cal P}(\alpha,\beta)\leq u$, where ${\cal P}(\alpha,\beta)$ represents the lowest possible value of $u$ for which the problem is feasible and all the constraints are satisfied. Given an interval $[u_{0},u_{1}]$ containing ${\cal P}(\alpha,\beta)$, the optimal detection rule $\boldsymbol{\delta}$ and the optimal estimation cost ${\cal P}(\alpha,\beta)$ can be determined by a bi-section search between $u_{0}$ and $u_{1}$ iteratively, solving the feasibility problem in each iteration. To solve the feasibility problem, we define an auxiliary convex optimization problem

[TABLE]

Algorithm 1 summarises the steps for determining ${\cal P}(\alpha,\beta)$.

Appendix C Proof of Theorem 4

To solve the problem in (103), a Lagrangian function is constructed according to

[TABLE]

where $\boldsymbol{\ell}\triangleq\left[\ell_{0},\dots,\ell_{T+2}\right]$ are the non-negative Lagrangian multipliers selected to satisfy the constraints in (24), such that

[TABLE]

The Lagrangian dual function is given by

[TABLE]

where

[TABLE]

and for $i\in\left\{1,\dots,T\right\}$,

[TABLE]

Therefore, the optimum detection rules that minimize $d(\boldsymbol{\ell})$ are given by:

[TABLE]

where

[TABLE]

Hence, the proof is concluded.

Appendix D Proof of Theorem 6

We can write ${\sf P}_{\sf is}(\boldsymbol{\hat{\delta}_{1}})$ as

[TABLE]

Note that $\mathbb{P}({\sf D}_{\sf is}\neq{\sf T}_{\sf is}\mid{\sf D}_{\sf d}=\hat{\sf H}_{1},{\sf T}_{\sf d}=\hat{{\sf H}}_{0})=1$ for any decision rule $\boldsymbol{\hat{\delta}_{1}}$, and the terms

${{\mathbb{P}}({\sf T}_{\sf d}=\hat{\sf H}_{1}\mid{\sf D}_{\sf d}=\hat{\sf H}_{1})}$ and ${\mathbb{P}}({\sf T}_{\sf d}=\hat{\sf H}_{0}\mid{\sf D}_{\sf d}=\hat{\sf H}_{1})$ are independent of the decision rule $\boldsymbol{\hat{\delta}}_{1}$. Therefore, minimizing ${\sf P}_{\sf is}(\boldsymbol{\hat{\delta}_{1}})$ is equivalent to minimizing $\mathbb{P}({\sf D}_{\sf is}\neq{\sf T}_{\sf is}\mid{\sf D}_{\sf d}=\hat{\sf H}_{1},{\sf T}_{\sf d}=\hat{{\sf H}}_{1})$. Furthermore,

[TABLE]

where $R_{i}\subseteq{\cal Y}^{n}$ is the region corresponding to the decision ${\sf D}_{\sf is}={\sf H}_{i}$ and $\hat{R}_{1}\subseteq{\cal Y}^{n}$ is the region corresponding to the decision ${\sf D}_{\sf d}=\hat{\sf H}_{1}$. Since we form the decision ${\sf D}_{\sf is}$ only when ${\sf D}_{\sf d}=\hat{{\sf H}}_{1}$, we conclude that $R_{i}\subseteq\hat{R}_{1}$ and hence, $R_{i}\cap\hat{R}_{1}=R_{i}$. Therefore,

[TABLE]

and $\mathbb{P}({\sf D}_{\sf is}\neq{\sf T}_{\sf is}\mid{\sf D}_{\sf d}=\hat{\sf H}_{1},{\sf T}_{\sf d})$ is minimized when the regions $R_{i}$ are chosen using the decision rule

[TABLE]

where

[TABLE]

Appendix E Proof of Theorem 7

For the decision rule defined in Theorem 7, we denote the decision formed as $\bar{{\sf D}}_{\sf is}\in\{{\sf H}_{1},\dots,{\sf H}_{T}\}$ to distinguish it from the decision formed by the optimal decision rule in Lemma 6. Also, from Lemma 1,

[TABLE]

Next, we find an upper bound on ${{\sf P}}_{\sf is}^{u}$. When $X$ is guessed to be $X_{c}$, the probability that the decision formed by the rule in Theorem 7 is ${\sf H}_{j}$ when the true model is ${\sf H}_{i}$ is given by ${\mathbb{P}}(\bar{\sf D}_{\sf is}={\sf H}_{j}\mid{\sf T}_{\sf is}={\sf H}_{i},X_{c})$. Assuming that the probabilities $\epsilon_{i}$, for $i\in\{1,\dots,T\}$, are independent of the choice of $X_{c}$, we have

[TABLE]

where we have defined

[TABLE]

Therefore,

[TABLE]

Let $T^{\prime}\triangleq\frac{T(T-1)}{2}$. Note that

[TABLE]

where $R_{ij}=\left\{{\bf{{Y}}}:\prod_{a\in S_{j}}{\sf LR}_{a}\geq\prod_{b\in S_{i}}{\sf LR}_{b}\right\}$. Define $B_{i}$ as the set of coordinates deemed to be compromised in ${\sf H}_{i}$ but not in ${\sf H}_{j}$, i.e., $B_{i}\triangleq S_{i}-S_{i}\cap S_{j}$, with $\lvert B_{i}\rvert=r_{i}$. Similarly, define $B_{j}\triangleq S_{j}-S_{i}\cap S_{j}$, with $\lvert B_{j}\rvert=r_{j}$. Also, since we have $g_{l}^{j}$, for $l\in\{1,\dots,n\},j\in\{0,1\}$, to be conditionally independent distributions given $X_{c}$, we have

[TABLE]

where $\bar{S}_{i}\triangleq\{1,\dots,m\}\backslash S_{i}$. Then, the region $R_{ij}$ is equivalent to

[TABLE]

Therefore,

[TABLE]

where ${\bf{{Y}}}_{B_{i}\cup B_{j}}$ is the data for the coordinates in the set $B_{i}\cup B_{j}$ and

[TABLE]

Using the inequality

[TABLE]

and

[TABLE]

we can modify the inequality in (127) as

[TABLE]

Following the similar line of analysis as in (127)-(133), it can be verified that

[TABLE]

The inequalities in (133) and (134) hold for all $\lambda\in[0,1]$. Hence, from (122), (133) and (134),

[TABLE]

Under the assumption that the pdfs of the random variables $Y_{a}(b)\mid X,$ belong to the location-scale family of distributions with infinite support and only the location parameter a function of $X$, we use the change of variables $\tilde{Y}_{a}(b)=Y_{a}(b)\mid X_{c}$ to obtain

[TABLE]

Note that

[TABLE]

By taking the limit ${n\rightarrow\infty}$ and using the monotonicity of the $\log$ function, we have

[TABLE]

Therefore,

[TABLE]

We now find a lower bound on ${\sf P}_{\sf is}^{\rm l}$. Under the perfect knowledge of $X$, we have

[TABLE]

Note that

[TABLE]

For further analysis, we adopt the approach used in [31]. Under model ${\sf H}_{i}$, the data points $Y_{w}$ are distributed according to the pdf ${f}_{i}$ , for $w\in\{1,\dots n\}$ and $i\in\{1,\dots T\}$. Define $b_{i,j}^{\lambda}(Y_{w})$ as the pdf

[TABLE]

and let

[TABLE]

It can be readily verified that

[TABLE]

where $D_{\sf KL}(f\|g)$ denotes the Kullback-Liebler divergence between distributions $f$ and $g$. Define the probability measure $\tilde{{\mathbb{P}}}$ as

[TABLE]

and define a random process $M_{r}$ as

[TABLE]

where $F_{w-1}$ is the $\sigma-$field generated by $\{Y_{1},\dots,Y_{w-1}\}$, and the expectation is with respect to the probability measure $\tilde{{\mathbb{P}}}$. It can be verified that the process $M_{r}$ is a stable martingale. According to the martingale stability theorem [32], we have

[TABLE]

This can be equivalently written in the following form:

[TABLE]

Since for a given $X$ the random vectors $Y_{w}$ are i.i.d., we have

[TABLE]

By using (154) and restructuring (153), we obtain

[TABLE]

Under the probability measure $\tilde{{\mathbb{P}}}$, we denote the probability of deciding ${\sf H}_{i}$ as the true model when $X$ is given as $\tilde{{\mathbb{P}}}({\sf D}_{\sf is}={\sf H}_{i}\mid X)$ and note that either $\tilde{{\mathbb{P}}}({\sf D}_{\sf is}\neq{\sf H}_{i}\mid X)\geq\frac{1}{2}$ or $\tilde{{\mathbb{P}}}({\sf D}_{\sf is}={\sf H}_{i}\mid X)\geq\frac{1}{2}$. Then, using the assumption that $\tilde{{\mathbb{P}}}({\sf D}_{\sf is}={\sf H}_{i}\mid X)\geq\frac{1}{2}$ holds, from (155) we get

[TABLE]

for any $\kappa>0$. Using (150), we conclude that (156) is equivalent to

[TABLE]

where $R$ is the region

[TABLE]

In the region $R$, we have

[TABLE]

where region $R_{2}$ is $\{{\sf D}_{\rm is}\neq{\sf H}_{j}\mid X\}$ and also, $R\subseteq R_{2}$. From (157) and (E), it is concluded that

[TABLE]

for any $\nu,\kappa>0$. Similarly, if the case $\tilde{{\mathbb{P}}}({\sf D}_{\rm is}\neq{\sf H}_{i}\mid X)\geq\frac{1}{2}$ holds,

[TABLE]

Using (E), (160) and (161), we have

[TABLE]

Using the fact that $\kappa$ and $\nu$ can be made arbitrarily close to [math], and using (149), we get

[TABLE]

where (166) follows from the assumption that the pdfs $g_{a}^{j}$ belong to the location-scale family of distributions and have infinite support. From (E), (165), it is clear that

[TABLE]

Then, using (118), we conclude that the error exponents of ${\sf P}_{\sf is}(\boldsymbol{\hat{\delta}_{1}})$ and ${\sf P}_{\sf is}(\boldsymbol{\bar{\delta}_{1}})$ are the same and are given by (166).

Appendix F Proof of Theorem 8

We aim to design the estimator $U_{l}^{j}$ and the decision rule $\boldsymbol{\bar{\delta}}_{l}$ that minimize the utility function ${J}_{l}^{j}(\delta_{l}^{j},\bar{\delta}_{l}^{\sf r},U_{l}^{j})$ subject to the constraint

[TABLE]

Since the effect of estimator only appears in ${{J}_{l}^{j}(\delta_{l}^{j},\bar{\delta}_{l}^{\sf r},U_{l}^{j})}$, the optimization problem can be decoupled similarly to the approach followed earlier. The optimal estimator can be readily verified to be

[TABLE]

where $\hat{X}_{l}^{j}({\bf{{Y}}}_{l})$ is defined in (71). In order to design the decision rules, we start by noting that for a decision rule $\boldsymbol{\bar{\delta}}_{l}$, such that,

[TABLE]

we can design another decision rule $\boldsymbol{{\Delta}}_{l}\triangleq[\Delta_{l}^{\sf r}({\bf{{Y}}}_{l}),\Delta_{l}^{\sf u}({\bf{{Y}}}_{l})]$, such that, $\int\delta_{l}^{j}({\bf{{Y}}}_{l}){\Delta}^{\sf r}_{l}({\bf{{Y}}}_{l})g_{l}^{j}({\bf{{Y}}}_{l}){\rm d}{\bf{{Y}}}_{l}=1-\nu_{l}^{j}$ with the same estimation performance. To show this, we set

[TABLE]

which satisfies (168) with equality. Using (170), note that ${\Delta}^{\sf r}_{l}({\bf{{Y}}}_{l})<\bar{\delta}^{\sf r}_{l}({\bf{{Y}}}_{l})$, which implies that $\boldsymbol{{\Delta}}_{l}$ is a valid decision rule. We can easily verify that

[TABLE]

which implies that the estimation performance is the same for both decision rules, and therefore, we can restrict our design for the optimum decision rule to the class of rules that satisfy (168) with equality. Under the equality condition, $\int\delta_{j}^{i}({\bf{{Y}}}_{i}),{\bar{\delta}}^{\sf r}_{l}({\bf{{Y}}}_{l})g_{l}^{j}({\bf{{Y}}}_{l}){\rm d}{\bf{{Y}}}_{l}=1-\nu_{l}^{j}$, in which case minimizing ${{{J}_{l}^{j}(\delta_{l}^{j},\bar{\delta}^{\sf r}_{l},\hat{X}_{l}^{j})}}$ is equivalent to minimizing $\int\delta_{l}^{j}({\bf{{Y}}}_{l})\bar{\delta}^{\sf r}_{l}({\bf{{Y}}}_{l})\hat{{\sf C}}_{l}^{j}({\bf{{Y}}}_{l})g_{l}^{j}({\bf{{Y}}}_{l}){\rm d}{\bf{{Y}}}_{l}$. Let $\gamma_{l}^{j}\geq 0$ be the solution to

[TABLE]

where region $\hat{R}$ is $\left\{{\bf{{Y}}}_{l}:\gamma_{l}^{j}\geq\hat{{\sf C}}_{l}^{j}({\bf{{Y}}}_{l})\right\}$. Then,

[TABLE]

Clearly, $\int\delta^{j}_{l}({\bf{{Y}}}_{l})\bar{\delta}^{\sf r}_{l}({\bf{{Y}}}_{l})\hat{{\sf C}}_{l}^{j}({\bf{{Y}}}_{l})g_{l}^{j}({\bf{{Y}}}_{l}){\rm d}{\bf{{Y}}}_{l}\geq\int_{\hat{R}}\delta^{j}_{l}({\bf{{Y}}}_{l})\hat{{\sf C}}_{l}^{j}({\bf{{Y}}}_{l})g_{l}^{j}({\bf{{Y}}}_{l}){\rm d}{\bf{{Y}}}_{l}$ and therefore, the decision rule $\bar{\delta}^{\sf r}_{l}({\bf{{Y}}}_{l})$ given by

[TABLE]

is optimal as it ensures optimal estimation performance and satisfies the constraint in (168).