Adversarial Source Identification Game with Corrupted Training

TL;DR

This paper analyzes a game-theoretic approach to source identification under adversarial conditions where training data can be corrupted, deriving equilibrium strategies and performance limits in the asymptotic regime.

Contribution

It introduces a novel equilibrium analysis for source identification games with corrupted training data, providing explicit performance bounds and decision strategies.

Findings

Derived the unique equilibrium of the adversarial game

Established the best achievable performance under exponential error decay

Analyzed source distinguishability as a function of corruption and distortion

Abstract

We study a variant of the source identification game with training data in which part of the training data is corrupted by an attacker. In the addressed scenario, the defender aims at deciding whether a test sequence has been drawn according to a discrete memoryless source $X \sim P_X$, whose statistics are known to him through the observation of a training sequence generated by $X$. In order to undermine the correct decision under the alternative hypothesis that the test sequence has not been drawn from $X$, the attacker can modify a sequence produced by a source $Y \sim P_Y$ up to a certain distortion, and corrupt the training sequence either by adding some fake samples or by replacing some samples with fake ones. We derive the unique rationalizable equilibrium of the two versions of the game in the asymptotic regime and by assuming that the defender bases its decision by relying only on the first order statistics of the test and the training sequences. By mimicking Stein's lemma, we derive the best achievable performance for the defender when the first type error probability is required to tend to zero exponentially fast with an arbitrarily small, yet positive, error exponent. We then use such a result to analyze the ultimate distinguishability of any two sources as a function of the allowed distortion and the fraction of corrupted samples injected into the training sequence.

Equations

$$\mathcal{D}(P||Q)=\sum _{a\in \mathcal{X}}P(a){\log }_2\frac{P(a)}{Q(a)}.$$

$$h(P_{x^n},P_{t^m})=\mathcal{D}(P_{x^n}||P_{r^{n+m}})+\frac{m}{n}\mathcal{D}(P_{t^m}||P_{r^{n+m}}),$$

$$d_{L_{p}}(P,Q)=\bigg{(}\sum_{a\in\mathcal{X}}|P(a)-Q(a)|^{p}\bigg{)}^{1/p}.$$

$$d(x,A)=\underset{a\in A}{\inf }d(a,x).$$

$${\delta }_H(A,B)=\max \{{\delta }_A(B),{\delta }_B(A)\}.$$

$${\text{SS}}_D=\{{\Lambda }^{n\times m}\subset {\mathcal{P}}^n\times {\mathcal{P}}^m:\underset{P_X\in \mathcal{P}}{\max }\underset{s\in {\text{SS}}_A}{\max }{P}_{fp}\le 2^{-\lambda n}\},$$

$$P_{t^m}=\alpha P_{{\tau }^{m_2}}+(1-\alpha )P_{{\tau }^{m_1}},$$

$$d(y^n,z^n)=\sum _{i,j}n(i,j)d(i,j),$$

$$\frac{d(y^n,z^n)}{n}=\sum _{i,j}{S}_{YZ}^n(i,j)d(i,j).$$

$$P_{z^n}(j)=\sum _i{S}_{YZ}^n(i,j)\triangleq S_Z^n(j).$$

$$S_Y^n=P_{y^n}$$

$$\sum _{i,j}{S}_{YZ}^n(i,j)d(i,j)\le L.$$

$${\text{SS}}_A={\text{SS}}_{A,T}\times {\text{SS}}_{A,O},$$

$${\text{SS}}_{A,T}$$

$${\text{SS}}_{A,O}$$

$${\text{SS}}_D=\{{\Lambda }^{n\times m}\subset {\mathcal{P}}^n\times {\mathcal{P}}^m:\underset{P_X}{\max }\underset{Q(\cdot )\in {\text{SS}}_{A,T}}{\max }{P}_{fp}\le 2^{-\lambda n}\}.$$

$$u({\Lambda }^{n\times m},(Q(\cdot ),S_{YZ}^n(\cdot ,\cdot )))=-P_{fn}.$$

$$\displaystyle\SS_{A,T}~{}=~{}\bigg{\{}Q(P_{\tau^{m_{1}}},P_{y^{n}}):~{}\mathcal{P}^{m_{1}}\times\mathcal{P}^{n}\rightarrow\mathcal{P}^{m_{2}}\bigg{\}}.$$

$${\text{SS}}_D=\{{\Lambda }^{n\times m}\subset {\mathcal{P}}^n\times {\mathcal{P}}^m:\underset{P_X}{\max }\underset{Q(\cdot ,\cdot )\in {\text{SS}}_{A,T}}{\max }{P}_{fp}\le 2^{-\lambda n}\}.$$

$${\text{SS}}_A={\text{SS}}_{A,T}\times {\text{SS}}_{A,O}$$

$$u({\Lambda }^{n\times m},(Q(\cdot ,\cdot ),S_{YZ}^n(\cdot ,\cdot )))=-P_{fn}.$$

$$P_{fp}=$$

$$=$$

$$\sum _{(P_{x^n},P_{t^m})\in {\bar{\Lambda }}^{n\times m}}{P}_X(T(P_{x^n}))\cdot \sum _{\begin{array}{c}{P}_{{\tau }^{m_1}}\in {\mathcal{P}}^{m_1}:\\ \alpha Q(P_{{\tau }^{m_1}},P_{y^n})+(1-\alpha )P_{{\tau }^{m_1}}=P_{t^m}\end{array}}{P}_X(T(P_{{\tau }^{m_1}})),$$

$${\text{SS}}_D$$

$$\displaystyle\sum_{(P_{x^{n}},P_{t^{m}})\in\bar{\Lambda}^{n\times m}}P_{X}(T(P_{x^{n}}))\cdot\sum_{\begin{subarray}{c}P_{\tau^{m_{1}}}\in\mathcal{P}^{m_{1}}:\\ \alpha Q(P_{\tau^{m_{1}}},P_{y^{n}})+(1-\alpha)P_{\tau^{m_{1}}}=P_{t^{m}}\end{subarray}}P_{X}(T(P_{\tau^{m_{1}}}))\leq 2^{-\lambda n}\bigg{\}}.$$

$${\Lambda }^{n\times m,\ast }=\left\{(P_{v^n},P_{t^m}):\underset{Q\in {\mathcal{P}}^{m_2}}{\min }h\left(P_{v^n},\frac{P_{t^m}-\alpha Q}{1-\alpha }\right)\le \lambda -{\delta }_n\right\}$$

$${\delta }_n=|\mathcal{X}|\frac{\log (n+1)((1-\alpha )nc+1)}{n},$$

$$\underset{P_X}{\max }\underset{Q(\cdot ,\cdot )}{\max }{P}_{fp}\le$$

$$\underset{P_X}{\max }\sum _{P_{y^n}\in {\mathcal{P}}^n}{P}_Y(T(P_{y^n}))\cdot \sum _{\begin{array}{c}(P_{x^n},P_{t^m})\\ \in {\bar{\Lambda }}^{n\times m,\ast }\end{array}}{P}_X(T(P_{x^n}))\cdot$$

$$\underset{Q(\cdot ,\cdot )}{\max }\sum _{\begin{array}{c}{P}_{{\tau }^{m_1}}\in {\mathcal{P}}^{m_1}:\\ \alpha Q(P_{{\tau }^{m_1}},P_{y^n})+(1-\alpha )P_{{\tau }^{m_1}}=P_{t^m}\end{array}}{P}_X(T(P_{{\tau }^{m_1}})).$$

$$P_X(T(P_{{\tau }^{m_1}}))\le \underset{Q\in {\mathcal{P}}^{m_2}}{\max }{P}_X\left(T\left(\frac{P_{t^m}-\alpha Q}{1-\alpha }\right)\right),$$

$$\underset{P_X}{\max }\underset{Q(\cdot ,\cdot )}{\max }{P}_{fp}\le$$

$$\underset{P_X}{\max }\sum _{P_{y^n}\in {\mathcal{P}}^n}{P}_Y(T(P_{y^n}))\cdot$$

Full text

I Introduction

Adversarial Signal Processing (AdvSP) is an emerging discipline aiming at modelling the interplay between a defender wishing to carry out a certain processing task, and an attacker aiming at impeding it [1]. Binary decision in an adversarial setup is one of the most recurrent problems in AdvSP, due to its importance in many application scenarios. Among binary decision problems, source identification is one of the most studied subjects, since it lies at the heart of several security-oriented disciplines, like multimedia forensics, anomaly detection, traffic monitoring, steganalysis and so on.

The source identification game has been introduced in [2] to model the interplay between the defender and the attacker by resorting to concepts drawn from game and information theory. According to the model put forward in [2], the defender and the attacker have a perfect knowledge of the to-be-distinguished sources. In [3] the analysis is pushed a step forward by considering a scenario in which the sources are known only through the observation of a training sequence. Finally, [4] introduces the security margin concept, a synthetic parameter characterising the ultimate distinguishability of two sources under adversarial conditions.

In this paper, we extend the analysis further, by considering a situation in which the attacker may interfere with the learning phase by corrupting part of the training sequence. Adversarial learning is a rather novel concept, which has been studied for some years from a machine learning perspective [5, 6, 7]. Due to the natural vulnerability of machine learning systems, in fact, the attacker may take an important advantage if no countermeasures are adopted by the defender. The use of a training sequence to gather information about the statistics of the to-be-distinguished sources can be seen as a very simple learning mechanism, and the analysis of the impact that an attack carried out in such a phase has on the performance of a decision system may help shedding new light on this important problem. To be specific, we extend the game-theoretic framework introduced in [3] and [4] to model a situation in which the attacker is given the possibility of corrupting part of the training sequence. By adopting a game-theoretic perspective, we derive the optimal strategy for the defender and the optimal corruption strategy for the attacker when the length of the training sequence and the observed sequence tends to infinity. Given such optimum strategies, expressed in the form of game equilibrium point, we analyse the best achievable performance when the type I and II error probabilities tend to zero exponentially fast. Specifically, we study the distinguishability of the sources as a function of the fraction of training samples corrupted by the attacker and when the test sequence can be modified up to a certain distortion level. The results of the analysis are summarised in terms of blinding corruption level, defined as the fraction of corrupted samples making a reliable distinction between the two sources impossible, and security margin, defined as the maximum distortion of the observed sequence for which a reliable distinction is possible (see [4]). The analysis is applied to two different scenarios wherein the attacker is allowed respectively to add a certain amount of fake samples to the training sequence and to selectively replace a fraction of the samples of the training sequences with fake samples. As we will see, the second case is more favourable to the attacker, since a lower distortion and a lower number of corrupted training samples are enough to prevent a correct decision.

Given the above general framework, the main results proven in this paper can be summarised as follows:

We rigorously define the source identification game with addition of corrupted training samples ($SI_{c\textnormal{-}tr}^{a}$ game) and show that such a game is a dominance solvable game admitting an asymptotic equilibrium point when the length of the training and test sequences tend to infinity (Theorem 1 and following discussion in Section III); 2. 2.

We evaluate the payoff of the game at the equilibrium and derive the expression of the indistinguishability region, defined as the region with the sources $Y$ which can not be distinguished from $X$ because of the attack (Theorems 2 and 3, Section III); 3. 3.

Given any two sources $X$ and $Y$, we derive the security margin and the blinding corruption level defined as the maximum distortion introduced into the test sequence and maximum fraction of fake training samples introduced by the attacker, still allowing the distinction of $X$ and $Y$ while ensuring positive error exponents for the two kinds of errors of the test (Theorem 4 and Definition 3 in Section V); 4. 4.

We repeat the entire analysis for the source identification game with selective replacement of training samples ($SI_{c\textnormal{-}tr}^{r}$ game), and compare the two versions of the game (Theorem 5 and subsequent discussion in Section VI). 5. 5.

The main proofs of the paper rely on a generalised version of Sanov’s theorem [8, 9], which is proven in Appendix -A. In fact, Theorem 6, and its use to simplify some of the proofs in the paper, can be seen as a further methodological contribution of our work.

This paper considerably extends the analysis presented in [10], by providing a formal proof of the results anticipated in [10]111We also give a more precise formulation of the problem, by correcting some inaccuracies present in [10]. and make a step forward by studying a more complex corruption scenario in which the attacker has the freedom to replace a given percentage of the training samples rather than simply adding some fake samples to the original training sequence.

The paper is organised as follows. Section II summarises the notation used throughout the paper, gives some definitions and introduces some basics concept of Game theory that will be used in the sequel. Section III gives a rigorous definition of the $SI_{c\textnormal{-}tr}^{a}$ game, explaining the rationale behind the various assumptions made in the definition. In Section IV, we prove the main theorems of the paper regarding the asymptotic equilibrium point of the $SI_{c\textnormal{-}tr}^{a}$ game and the payoff at the equilibrium. Section V leverages on the results proven in Section IV to introduce the concepts of blind corruption level and security margin, and evaluating them in the setting provided by the $SI_{c\textnormal{-}tr}^{a}$ game. Section VI, introduces and solves the $SI_{c\textnormal{-}tr}^{r}$ game, by paying attention to compare the results of the analysis with the corresponding results of the $SI_{c\textnormal{-}tr}^{a}$ game. The paper ends in Section VII, with a summary of the main results proven in the paper and the description of possible directions for future work. In order to avoid burdening the main body of the paper, the most technical details of the proofs are gathered in the Appendix.

II Notation and definitions

In this section, we introduce the notation and definitions used throughout the paper. We will use capital letters to indicate discrete memoryless sources (e.g. $X$). Sequences of length $n$ drawn from a source will be indicated with the corresponding lowercase letters (e.g. $x^{n}$); accordingly, $x_{i}$ will denote the $i\textnormal{-}$th element of a sequence $x^{n}$. The alphabet of an information source will be indicated by the corresponding calligraphic capital letter (e.g. $\mathcal{X}$). The probability mass function (pmf) of a discrete memoryless source $X$ will be denoted by $P_{X}$. The calligraphic letter $\mathcal{P}$ will be used to indicate the class of all the probability mass functions, namely, the probability simplex in $\mathds{R}^{|\mathcal{X}|}$. The notation $P_{X}$ will be also used to indicate the probability measure ruling the emission of sequences from a source $X$, so we will use the expressions $P_{X}(a)$ and $P_{X}(x^{n})$ to indicate, respectively, the probability of symbol $a\in\mathcal{X}$ and the probability that the source $X$ emits the sequence $x^{n}$, the exact meaning of $P_{X}$ being always clearly recoverable from the context wherein it is used. We will use the notation $P_{X}(A)$ to indicate the probability of $A$ (be it a subset of $\mathcal{X}$ or $\mathcal{X}^{n}$) under the probability measure $P_{X}$. Finally, the probability of a generic will be denoted by $Pr\{\}$.

Our analysis relies extensively on the concepts of type and type class defined as follows (see [8] and [11] for more details). Let $x^{n}$ be a sequence with elements belonging to a finite alphabet $\mathcal{X}$. The type $P_{x^{n}}$ of $x^{n}$ is the empirical pmf induced by the sequence $x^{n}$, i.e. $\forall a\in\mathcal{X},P_{x^{n}}(a)=\frac{1}{n}\sum_{i=1}^{n}\delta(x_{i},a)$, where $\delta(x_{i},a)=1$ if $x_{i}=a$ and zero otherwise. In the following, we indicate with $\mathcal{P}^{n}$ the set of types with denominator $n$, i.e. the set of types induced by sequences of length $n$. Given $P\in\mathcal{P}^{n}$, we indicate with $T(P)$ the type class of $P$, i.e. the set of all the sequences in $\mathcal{X}^{n}$ having type $P$. We denote by $\mathcal{D}(P||Q)$ the Kullback-Leibler (KL) divergence between two distributions $P$ and $Q$, defined on the same finite alphabet $\mathcal{X}$ [8]:

[TABLE]

Most of our results are expressed in terms of the generalised log-likelihood ratio function $h$ (see [3, 12, 13]), which for any two given sequences $x^{n}$ and $t^{m}$ is defined as:

[TABLE]

where $P_{r^{n+m}}$ denotes the type of the sequence $r^{n+m}$, obtained by concatenating $x^{n}$ and $t^{m}$, i.e. $r^{n+m}=x^{n}\|t^{m}$. The intuitive meaning behind the above definition is that $P_{r^{n+m}}$ is the pmf which maximises the probability that a memoryless source generates two independent sequences belonging to $T(P_{x^{n}})$ and $T(P_{t^{m}})$, and that such a probability is equal to $2^{-nh(P_{x^{n}},P_{t^{m}})}$ at the first order in the exponent (see [13] or Lemma 1 in [3]).

Throughout the paper, we will need to compute limits and distances in $\mathcal{P}$. We can do so by choosing one of the many available distances defined over $\mathbb{R}^{|\mathcal{X}|}$ and for which $\mathcal{P}$ is a bounded set, for instance the $L_{p}$ distance for which we have:

[TABLE]

Without loss of generality, we will prove all our results by adopting the $L_{1}$ distance, the generalisation to different $L_{p}$ metrics being straightforward. In the sequel, distances between pmf’s in $\mathcal{P}$ will be simply indicated as $d(\cdot,\cdot)$ as a shorthand for $d_{L_{1}}(\cdot,\cdot)$222Throughout the paper, we will use the symbol $d(\cdot,\cdot)$ to indicate both the distortion between two sequences in $\mathcal{X}^{n}$ and the $L_{1}$ distance between two pmf’s in $\mathcal{P}$, the exact meaning being always clear from the context,.

We also need to introduce the Hausdorff distance as a way to measure distances between subsets of a metric space [14]. Let $S$ be a generic space and $d$ a distance measure defined over $S$. For any point $x\in S$ and any non-empty subset $A\subseteq S$, the distance of $x$ from the subset $A$ is defined as:

[TABLE]

Given the above definition, the Hausdorff distance between any two subsets of $S$ is defined as follows.

Definition 1**.**

For any two subsets $A$ and $B$ of $S$, let us define $\delta_{B}(A)=\sup_{b\in B}d(b,A)$. The Hausdorff distance $\delta_{H}(A,B)$ between $A$ and $B$ is given by:

[TABLE]

If the sets $A$ and $B$ are bounded with respect to $d$, then the Hausdorff distance always takes a finite value. The Hausdorff distance does not define a true metric, but only a pseudometric, since $\delta_{H}(A,B)=0$ implies that the closures of the sets $A$ and $B$ coincide, namely $\text{\em cl}(A)=\text{\em cl}(B)$, but not necessarily that $A=B$. For this reason, in order for $\delta_{H}$ to be a metric, we need to restrict its definition to closed subsets333Note that in this case the $\inf$ and $\sup$ operations involved in the definition of the Hausdorff distance can be replaced with $\min$ and $\max$, respectively.. Let then $\mathcal{L}(S)$ denote the space of non-empty closed and limited subsets of $S$ and let $\delta_{H}:\mathcal{L}(S)\times\mathcal{L}(S)\rightarrow[0,\infty)$. Then, the space $\mathcal{L}(S)$ endowed with the Hausdorff distance is a metric space [15] and we can give the following definition:

Definition 2**.**

Let $\{K_{n}\}$ be a sequence of closed and limited subsets of $S$, i.e., $K_{n}\in\mathcal{L}(S)$ $\forall n$. We use the notation $K_{n}\overset{H}{\rightarrow}~{}K$ to indicate that the sequence has limit in $(\mathcal{L}(S),\delta_{H})$ and the limiting set is $K$.

II-A Basic notions of Game Theory

In this section, we introduce some basic notions and definitions of Game Theory.

A 2-player game is defined as a quadruple $(\SS_{1},\SS_{2},u_{1},u_{2})$, where $\SS_{1}=\{s_{1,1}\dots s_{1,n_{1}}\}$ and $\SS_{2}=\{s_{2,1}\dots s_{2,n_{2}}\}$ are the set of strategies the first and the second player can choose from, and $u_{l}(s_{1,i},s_{2,j}),l=1,2$, is the payoff of the game for player $l$, when the first player chooses the strategy $s_{1,i}$ and the second chooses $s_{2,j}$. A pair of strategies $(s_{1,i},s_{2,j})$ is called a profile. When $u_{1}(s_{s1,i},s_{2,j})=-u_{2}(s_{1,i},s_{2,j})$, the win of a player is equal to the loss of the other and the game is said to be a zero-sum game. The sets $\SS_{1}$, $\SS_{2}$ and the payoff functions are assumed to be known to both players. Throughout the paper we consider strategic games, i.e., games in which the players choose their strategies beforehand without knowing the strategy chosen by the opponent player.

The final goal of game theory is to determine the existence of equilibrium points, i.e. profiles that in some sense represent the best choice for both players [16]. The most famous notion of equilibrium is due to Nash. A profile is said to be a Nash equilibrium if no player can improve its payoff by changing its strategy unilaterally. Despite its popularity, the practical meaning of Nash equilibrium is often unclear, since there is no guarantee that the players will end up playing at the equilibrium. A particular kind of games for which stronger forms of equilibrium exist are the so called dominance solvable games [16]. To be specific, a strategy is said to be strictly dominant for one player if it is the best strategy for the player, i.e., the strategy which corresponds to the largest payoff, no matter how the other player decides to play. When one such strategy exists for one of the players, he will surely adopt it. In a similar way, we say that a strategy $s_{l,i}$ is strictly dominated by strategy $s_{l,j}$, if the payoff achieved by player $l$ choosing $s_{l,i}$ is always lower than that obtained by playing $s_{l,j}$ regardless of the choice made by the other player. The recursive elimination of dominated strategies is a common technique for solving games. In the first step, all the dominated strategies are removed from the set of available strategies, since no rational player would ever play them. In this way, a new, smaller game is obtained. At this point, some strategies, that were not dominated before, may be dominated in the remaining game, and hence are eliminated. The process goes on until no dominated strategy exists for any player. A rationalizable equilibrium is any profile which survives the iterated elimination of dominated strategies [17, 18]. If at the end of the process only one profile is left, the remaining profile is said to be the only rationalizable equilibrium of the game. The corresponding strategies are the only rational choice for the two players and the game is said dominance solvable.

III Source identification game with addition of corrupted training samples ($SI_{c\textnormal{-}tr}^{a}$)

In this section, we give a rigorous definition of the Source Identification game with addition of corrupted training samples.

Given a discrete and memoryless source $X\sim P_{X}$ and a test sequence $v^{n}$, the goal of the defender (D) is to decide whether $v^{n}$ has been drawn from $X$ (hypothesis $H_{0}$) or not (alternative hypothesis $H_{1}$). By adopting a Neyman-Pearson perspective, we assume that D must ensure that the false positive error probability ($P_{fp}$), i.e., the probability of rejecting $H_{0}$ when $H_{0}$ holds (type I error) is lower than a given threshold. Similarly to the previous versions of the game studied in [2] and [3], we assume that D relies only on first order statistics to make a decision. For mathematical tractability, likewise earlier papers, we study the asymptotic version of the game when $n\tiny\to\infty$, by requiring that $P_{fp}$ decays exponentially fast when $n$ increases, with an error exponent at least equal to $\lambda$, i.e. $P_{fp}\leq 2^{-n\lambda}$. On its side, the attacker aims at increasing the false negative error probability ($P_{fn}$), i.e., the probability of accepting $H_{0}$ when $H_{1}$ holds (type II error). Specifically, A takes a sequence $y^{n}$ drawn from a source $Y\sim P_{Y}$ and modifies it in such a way that D decides that the modified sequence $z^{n}$ has been generated by $X$. In doing so, A must respect a distortion constraint requiring that the average per-letter distortion between $y^{n}$ and $z^{n}$ is lower than $L$.

Players A and D know the statistics of $X$ through a training sequence, however the training sequence can be partly corrupted by A. Depending on how the training sequence is modified by the attacker, we can define different versions of the game. In this paper, we focus on two possible cases: in the first case, hereafter referred to as source identification game with addition of corrupted samples $SI_{c\textnormal{-}tr}^{a}$, the attacker can add some fake samples to the original training sequence. In the second case, analysed in Section VI, the attacker can replace some of the training samples with fake values (source identification game with replacement of training samples - $SI_{c\textnormal{-}tr}^{r}$). It is worth stressing that, even if the goal of the attacker is to increase the false negative error probability, the training sequence is corrupted regardless of whether $H_{0}$ or $H_{1}$ holds, hence, in general, this part of the attack also affects the false positive error probability. As it will be clear later on, this forces the defender to adopt a worst case perspective to ensure that $P_{fp}$ is surely lower than $2^{-{\lambda n}}$.

As to $Y$, we assume that the attacker knows $P_{Y}$ exactly. For a proper definition of the payoff of the game, we also assume that D knows $P_{Y}$. This may seem a too strong assumption, however we will show later on that the optimum strategy of D does not depend on $P_{Y}$, thus allowing us to relax the assumption that D knows $P_{Y}$.

With the above ideas in mind, we are now ready to give a formal definition of the $SI_{c\textnormal{-}tr}^{a}$ game.

III-A Structure of the $SI_{c\textnormal{-}tr}^{a}$ game

A schematic representation of the $SI_{c\textnormal{-}tr}^{a}$ game is given in Figure 1.

Let $\tau^{m_{1}}$ be a sequence drawn from $X$. We assume that $\tau^{m_{1}}$ is accessible to A, who corrupts it by concatenating to it a sequence of fake samples $\tau^{m_{2}}$. Then A reorders the overall sequence in a random way so to hide the position of the fake samples. Note that reordering does not alter the statistics of the training sequence since the sequence is supposed to be generated from a memoryless source444By using the terminology introduced in [6], the above scenario can be referred to as a causative attack with control over training data.. In the following, we denote by $m$ the final length of the training sequence ($m=m_{1}+m_{2}$), and by $\alpha=\frac{m_{2}}{m_{1}+m_{2}}$ the portion of fake samples within it. The corrupted training sequence observed by D is indicated by $t^{m}$. Eventually, we hypothesize a linear relationship between the lengths of the test and the corrupted training sequence, i.e. $m=cn$, for some constant value $c$555In this paper, we are interested in studying the equilibrium point of the source identification game when the length of the test and training sequences tend to infinity. Strictly speaking, we should ensure that when $n$ grows, all the quantities $m$, $m_{1}$ and $m_{2}$ are integer numbers for the given $c$ and $\alpha$. In practice, we will neglect such an issue, since when $n$ grows the ratios $m/n$ and $m_{1}/(m_{1}+m_{2})$ can approximate any real values $c$ and $\alpha$. More rigorously, we could consider only rational values of $c$ and $\alpha$, and focus on subsequences of $n$ including only those values for which $m/n=c$ and $m_{1}/(m_{1}+m_{2})=\alpha$..

The goal of D is to decide if an observed sequence $v^{n}$ has been drawn from the same source that generated $t^{m}$ ($H_{0}$) or not ($H_{1}$). We assume that D knows that a certain percentage of samples in the training sequence are corrupted, but he has no clue about the position of the corrupted samples. The attacker can also modify the sequence generated by $Y$ so to induce a decision error. The corrupted sequence is indicated by $z^{n}$. With regard to the two phases of the attack, we assume that A first corrupts the training sequence, then he modifies the sequence $y^{n}$. This means that, in general, $z^{n}$ will depend both on $y^{n}$ and $t^{m}$, while $t^{m}$ (noticeably $\tau^{m_{2}}$) does not depend on $y^{n}$. Stated in another way, the corruption of the training sequence can be seen as a preparatory part of the attack, whose goal is to ease the subsequent camouflage of $y^{n}$.

For a formal definition of the $SI_{c\textnormal{-}tr}^{a}$ game, we must define the set of strategies available to D and A (respectively $\SS_{D}$ and $\SS_{A}$) and the corresponding payoffs.

III-B Defender’s strategies

The basic assumption behind the definition of the space of strategies available to D is that to make his decision D relies only on the first order statistics of $v^{n}$ and $t^{m}$. This assumption is equivalent to requiring that the acceptance region for hypothesis $H_{0}$, hereafter referred to as $\Lambda^{n\times m}$, is a union of pairs of type classes666We use the superscript $n\times m$ to indicate explicitly that $\Lambda^{n\times m}$ refers to $n$-long test sequences and $(m=cn)$-long training sequences., or equivalently, pairs of types $(P,R)$, where $P\in\mathcal{P}^{n}$ and $R\in\mathcal{P}^{m}$. To define $\Lambda^{n\times m}$, D follows a Neyman-Pearson approach, requiring that the false positive error probability is lower than a certain threshold. Specifically, we require that the false positive error probability tends to zero exponentially fast with a decay rate at least equal to $\lambda$. Given that the pmf $P_{X}$ ruling the emission of sequences under $H_{0}$ is not known and given that the corruption of the training sequence is going to impair D’s decision under $H_{0}$, we adopt a worst case approach and require that the constraint on the false positive error probability holds for all possible $P_{X}$ and for all the possible strategies available to the attacker. Given the above setting, the space of strategies available to D is defined as follows:

[TABLE]

where the inner maximization is performed over all the strategies available to the attacker. We will refine this definition at the end of the next section, after the exact definition of the space of strategies of the attacker.

III-C Attacker’s strategies

With regard to A, the attack consists of two parts. Given a sequence $y^{n}$ drawn from $P_{Y}$, and the original training sequence $\tau^{m_{1}}$, the attacker first generates a sequence of fake samples $\tau^{m_{2}}$ and mixes them up with those in $\tau^{m_{1}}$ producing the training sequence $t^{m}$ observed by D. Then he transforms $y^{n}$ into $z^{n}$, eventually trying to generate a pair of sequences ($z^{n},t^{m}$)777While reordering is essential to hide the position of fake samples to D, it does not have any impact on the position of ($z^{n},t^{m}$) with respect to $\Lambda^{n\times m}$, since we assumed that the defender bases its decision only on the first order statistic of the observed sequences. For this reason, we omit to indicate the reordering operator $\sigma$ in the attacking procedure. whose types belong to $\Lambda^{n\times m}$. In doing so, he must ensure that $d(y^{n},z^{n})\leq nL$ for some distortion function $d$.

Let us consider the corruption of the training sequence first. Given that the defender bases his decision only on the type of $t^{m}$, we are only interested in the effect that the addition of the fake samples has on $P_{t^{m}}$. By considering the different length of $\tau^{m_{1}}$ and $\tau^{m_{2}}$, we have:

[TABLE]

where $P_{t^{m}}\in\mathcal{P}^{m}$, $P_{\tau^{m_{1}}}\in\mathcal{P}^{m_{1}}$ and $P_{\tau^{m_{2}}}\in\mathcal{P}^{m_{2}}$. The first part of the attack, then, is equivalent to choosing a pmf in $\mathcal{P}^{m_{2}}$ and mixing it up with $P_{\tau^{m_{1}}}$. By the same token, it is reasonable to assume that the choice of the attacker depends only on $P_{\tau^{m_{1}}}$ rather than on the single sequence $\tau^{m_{1}}$. Arguably, the best choice of the pmf in $\mathcal{P}^{m_{2}}$ will depend on $P_{Y}$, since the corruption of the training sequence is instrumental in letting the defender think that a sequence generated by $Y$ has been drawn by the same source that generated $t^{m}$.

To describe the part of the attack applied to the test sequence, we follow the approach used in [4] based on transportation theory [19]. Let us indicate by $n(i,j)$ the number of times that the $i$-th symbol of the alphabet is transformed into the $j$-th one as a consequence of the attack. Similarly, let $S^{n}_{YZ}(i,j)=n(i,j)/n$ be the relative frequency with which such a transformation occurs. In the following, we refer to $S^{n}_{YZ}$ as transportation map. For any additive distortion measure, the distortion introduced by the attack can be expressed in terms of $n(i,j)$ and $S^{n}_{YZ}$. In fact, we have:

[TABLE]

[TABLE]

where $d(i,j)$ is the distortion introduced when symbol $i$ is transformed into symbol $j$.

The map $S^{n}_{YZ}$ also determines the type of the attacked sequence. In fact, by indicating with $P_{z^{n}}(j)$ the relative frequency of symbol $j$ into $z^{n}$, we have:

[TABLE]

Finally, we observe that the attacker can not change more symbols than there are in the sequence $y^{n}$; as a consequence a map $S^{n}_{YZ}$ can be applied to a sequence $y^{n}$ only if $S^{n}_{Y}(i)\triangleq\sum_{j}S^{n}_{YZ}(i,j)=P_{y^{n}}(i)$. Sometimes, we find convenient to explicit the dependence of the map chosen by the attacker on the type of $t^{m}$ and $y^{n}$, and hence we will also adopt the notation $S^{n}_{YZ}(P_{t^{m}},P_{y^{n}})$.

By remembering that $\Lambda^{n\times m}$ depends on $v^{n}$ only through its type, and given that the type of the attacked sequence depends on $S^{n}_{Y}$ only through $S^{n}_{YZ}$, we can define the second phase of the attack as the choice of a transportation map among all admissible maps, a map being admissible if:

[TABLE]

Hereafter, we will refer to the set of admissible maps as $\mathcal{A}^{n}(L,P_{y^{n}})$.

With the above ideas in mind, the set of strategies of the attacker can be defined as follows:

[TABLE]

where $\SS_{A,T}$ and $\SS_{A,O}$ indicate, respectively, the part of the attack affecting the training sequence and the observed sequence, and are defined as:

[TABLE]

Note that the first part of the attack ($\SS_{A,T}$) is applied regardless of whether $H_{0}$ or $H_{1}$ holds, while the second part ($\SS_{A,O}$) is applied only under $H_{1}$. We also stress that the choice of $Q(P_{\tau^{m_{1}}})$ depends only on the training sequence $\tau^{m_{1}}$, while the transportation map used in the second phase of the attack is a function of both on $y^{n}$ and $\tau^{m_{1}}$ (through $t^{m}$). Finally, we observe that with these definitions, the set of strategies of the defender can be redefined by explicitly indicating that the constraint on the false positive error probability must be verified for all possible choices of $Q(\cdot)\in\SS_{A,T}$, since this is the only part of the attack affecting $P_{fp}$. Specifically, we can rewrite (6) as

[TABLE]

III-D Payoff

The payoff is defined in terms of the false negative error probability, namely:

[TABLE]

Of course, D aims at maximising $u$ while A wants to minimise it.

III-E The $SI_{c\textnormal{-}tr}^{a}$ game with targeted corruption ($SI_{c\textnormal{-}tr}^{a,t}$ game)

The $SI_{c\textnormal{-}tr}^{a}$ game is difficult to solve directly, because of the 2-step attacking strategy. We will work around this difficulty by tackling first with a slightly different version of the game, namely the source identification game with targeted corruption of the training sequence, $SI_{c\textnormal{-}tr}^{a,t}$, depicted in Fig. 2.

Whereas the strategies available to the defender remain the same, for the attacker, the choice of $Q(\cdot)$ is targeted to the counterfeiting of a given sequence $y^{n}$. In other words, we will assume that the attacker corrupts the training sequence $\tau^{m_{1}}$ to ease the counterfeiting of a specific sequence $y^{n}$ rather than to increase the probability that the second part of the attack succeeds. This means that the part of the attack aiming at corrupting the training sequence also depend on $y^{n}$, that is:

[TABLE]

Even if this setup is not very realistic and is more favourable to the attacker, who can exploit the exact knowledge of $y^{n}$ (rather than its statistical properties) also for the corruption of the training sequence, in the next section we will show that, for large $n$, the $SI_{c\textnormal{-}tr}^{a,t}$ game is equivalent to the non-targeted version of the game we are interested in.

With the above ideas in mind, the $SI_{c\textnormal{-}tr}^{a,t}$ game is formally defined as follows.

III-E1 Defender’s strategies

[TABLE]

III-E2 Attacker’s strategies

[TABLE]

with $\SS_{A,T}$ and $\SS_{A,O}$ defined as in (17) and (14) respectively.

III-E3 Payoff

The payoff is still equal to the false negative error probability:

[TABLE]

IV Asymptotic equilibrium and payoff of the $SI_{c\textnormal{-}tr}^{a,t}$ and $SI_{c\textnormal{-}tr}^{a}$ games

In this section, we derive the asymptotic equilibrium point of the $SI_{c\textnormal{-}tr}^{a,t}$ and the $SI_{c\textnormal{-}tr}^{a}$ games when the length of the test and training sequences tends to infinity and evaluate the payoff at the equilibrium.

IV-A Optimum defender’s strategy

We start by deriving the asymptotically optimum strategy for D. As we will see, a dominant and universal strategy with respect to $P_{Y}$ exists for D. In other words, the optimum choice of D depends on neither the strategy chosen by the attacker nor $P_{Y}$. In addition, since the constraint on the false positive probability must be satisfied for all attackers’ strategy, the optimum strategy for the defender is the same for both the targeted and non-targeted versions of the game.

As a first step, we look for an explicit expression of the false positive error probability. Such a probability depends on $P_{X}$ and on the strategy used by A to corrupt the training sequence. In fact, the mapping of $y^{n}$ into $z^{n}$ does not have any impact on D’s decision under $H_{0}$. We carry out our derivations by focusing on the game with targeted corruption. It will be clear from our analysis that the dependence on $y^{n}$ has no impact on $P_{fp}$, and hence the same results hold for the game with non-targeted corruption.

For a given $P_{X}$ and $Q(\cdot,\cdot)$, $P_{fp}$ is equal to the probability that $Y$ generates a sequence $y^{n}$ and $X$ generates two sequences $x^{n}$ and $\tau^{m_{1}}$, such that the pair of type classes $(P_{x^{n}},\alpha Q(P_{\tau^{m_{1}}},P_{y^{n}})+(1-\alpha)P_{\tau^{m_{1}}})$ falls outside $\Lambda^{n\times m}$. Such a probability can be expressed as:

[TABLE]

where $\bar{\Lambda}^{n\times m}$ is the complement of $\Lambda^{n\times m}$, and where we have exploited the fact that under $H_{0}$ the training sequence $\tau^{m_{1}}$ and the test sequence $x^{n}$ are generated independently by $X$. Given the above formulation, the set of strategies available to D can be rewritten as:

[TABLE]

We are now ready to prove the following lemma, which describes the asymptotically optimum strategy for the defender for both versions of the game.

Lemma 1**.**

Let $\Lambda^{n\times m,*}$ be defined as follows:

[TABLE]

with

[TABLE]

where $|\mathcal{X}|$ is the cardinality of the source alphabet and where the minimisation over $Q$ is limited to all the $Q$’s such that $P_{t^{m}}-\alpha Q$ is nonnegative for all the symbols in $\mathcal{X}$.

Then:

$\max\limits_{P_{X}}\max\limits_{s\in\SS_{A}}~{}P_{fp}~{}\leq~{}2^{-n(\lambda-\nu_{n})}$, with $\lim\limits_{n\tiny\to\infty}\nu_{n}=0$, 2. 2.

$\forall\Lambda^{n\times m}~{}\in~{}\SS_{D}$, we have $\bar{\Lambda}^{n\times m}\subseteq{\bar{\Lambda}^{n\times m,*}}$.

Proof.

To prove the first part of the lemma, we see that from the expression of the false positive error probability given by eq. (IV-A), we can write:

[TABLE]

Let us consider the term within the inner summation. For each $P_{\tau^{m_{1}}}$ such that $\alpha Q(P_{\tau^{m_{1}}},P_{y^{n}})+(1-\alpha)P_{\tau^{m_{1}}}=P_{t^{m}}$, we have888It is easy to see that the bound (27) holds also for the non-targeted game, when $Q$ depends on the training sequence only ($Q(P_{\tau^{m_{1}}})$).:

[TABLE]

with the understanding that the maximisation is carried out only over the $Q$’s such that $P_{t^{m}}-\alpha Q$ is nonnegative for all the symbols in $\mathcal{X}$.

Thanks to the above observation, we can upper bound the false positive error probability as follows:

[TABLE]

where in $(a)$ we exploited the fact that the rest of the expression no longer depends on $P_{y^{n}}$. From this point, the proof goes along the same line of the proof of Lemma 2 in [3], by observing that $\max_{P_{X}}P_{X}(T(P_{x^{n}}))P_{X}\left(T\left(\frac{P_{t^{m}}-\alpha Q}{1-\alpha}\right)\right)$is upper bounded by $2^{-nh(P_{x^{n}},\frac{P_{t^{m}}-\alpha Q}{1-\alpha})}$, and that for each pair of types in $\bar{\Lambda}^{n\times m,*}$, $h(P_{x^{n}},\frac{P_{t^{m}}-\alpha Q}{1-\alpha})$ is larger than $\lambda-\delta_{n}$ for every $Q$ by the very definition of $\Lambda^{n\times m,*}$.

We now pass to the second part of the lemma. Let $\Lambda^{n\times m}$ be a strategy in $\SS_{D}$, and let $(P_{x^{n}},P_{t^{m}})$ be a pair of types contained in $\bar{\Lambda}^{n\times m}$. Given that $\Lambda^{n\times m}$ is an admissible decision region (see (18)), the probability that $X$ emits a test sequence belonging to $T(P_{x^{n}})$ and a training sequence $\tau^{m_{1}}$ such that after the attack $(\tau^{m_{1}}||\tau^{m_{2}})\in T(P_{t^{m}})$ must be lower than $2^{-\lambda n}$ for all $P_{X}$ and all possible attacking strategies, that is:

[TABLE]

where ${(a)}$ is obtained by replacing the maximisation over all possible strategies $Q(\cdot,\cdot)$, with a maximisation over $Q(\cdot,P_{y^{n}})$ for each specific $P_{y^{n}}$, and $(b)$ is obtained by considering only one term $P_{\tau^{m_{1}}}$ of the inner summation and optimising $Q(P_{\tau^{m_{1}}},P_{y^{n}})$ for that term. Finally, $(c)$ follows by observing that the optimum $Q(\cdot,P_{y^{n}})$ is the same for any $P_{y^{n}}$. As usual, the maximization over $Q$ in the last expression is restricted to the $Q$’s for which $P_{t^{m}}-\alpha Q~{}\geq~{}0$ for all the symbols in $\mathcal{X}$ 999It is easy to see that the same lower bound can be derived also for the non targeted case, as the optimum $Q$ in the second to last expression does not depend on $P_{y^{n}}$.

By lower bounding the probability that a memoryless source $X$ generates a sequence belonging to a certain type class (see [8], chapter 12), we can continue the above chain of inequalities as follows

[TABLE]

where $(a)$ derives from the minimization properties of the generalised log-likelihood ratio function $h()$ (see Lemma 1, in [3]). By taking the $\log$ of both terms we have:

[TABLE]

thus completing the proof of the lemma. ∎

Lemma 1 shows that the strategy $\Lambda^{n\times m,*}$ is asymptotically admissible (point 1) and optimal (point 2), regardless of the attack. From a game-theoretic perspective, this means that such a strategy is a dominant strategy for D and implies that the game is dominance solvable [17]. Similarly, the optimum strategy is a semi-universal one, since it depends on $P_{X}$ but it does not depend on $P_{Y}$.

It is clear from the proof of Lemma 1 that the same optimum strategy holds for the targeted and non-targeted versions of the game. The situation is rather different with regard to the optimum strategy for the attacker. Despite the existence of a dominant strategy for the defender, in fact, the identification of the optimum attacker’s strategy for the $SI_{c\textnormal{-}tr}^{a}$ game is not easy due to the 2-step nature of the attack. For this reason, in the following sections, we will focus on the targeted version of the game, which is easier to study. We will then use the results obtained for the $SI_{c\textnormal{-}tr}^{a,t}$ game to derive the best achievable performance for the case of non-targeted attack.

IV-B The $SI_{c\textnormal{-}tr}^{a,t}$ game: optimum attacker’s strategy and equilibrium point

Given the dominant strategy of D, for any given $\tau^{m_{1}}$ and $y^{n}$, the optimum attacker’s strategy for the $SI_{c\textnormal{-}tr}^{a,t}$ game boils down to the following double minimisation:

[TABLE]

where $P_{z^{n}}$ is obtained by applying the transformation map $S^{n}_{YZ}$ to $P_{y^{n}}$, and where $P_{t^{m}}=(1-\alpha)P_{\tau^{m_{1}}}+\alpha Q$. As usual, the minimisation over $Q^{\prime}$ is limited to the $Q^{\prime}$ such that all the entries of the resulting pmf are nonnegative.

As a remark, for $L=0$ (corruption of the training sequence only), we get:

[TABLE]

while, for $\alpha=0$ (classical setup, without corruption of the training sequence) we have:

[TABLE]

falling back to the known case of source identification with uncorrupted training, already studied in [3]. Having determined the optimum strategies of both players, it is immediate to state the following:

Theorem 1**.**

The $SI_{c\textnormal{-}tr}^{a,t}$ game is a dominance solvable game, whose only rationalizable equilibrium corresponds to the profile $(\Lambda^{n\times m,*},(Q^{*}(P\cdot,\cdot),~{}S^{n,*}_{YZ}(\cdot,\cdot))$.

Proof.

The theorem is a direct consequence of the fact that $\Lambda^{n\times m,*}$ is a dominant strategy for D. ∎

We remind that the concept of rationalizable equilibrium is much stronger than the usual notion of Nash equilibrium, since the strategies corresponding to such an equilibrium are the only ones that two rational players may adopt [16, 17].

IV-C The $SI_{c\textnormal{-}tr}^{a,t}$ game: payoff at the equilibrium

In this section, we derive the asymptotic value of the payoff at the equilibrium, to see who and under which conditions is going to win the game.

To start with, we identify the set of pairs $(P_{y^{n}},P_{\tau^{m_{1}}})$ for which, as a consequence of A’s action, D accepts $H_{0}$:

[TABLE]

If we fix the type of the non-corrupted training sequence ($P_{\tau^{m_{1}}}$), we obtain:

[TABLE]

where $\Lambda^{n,*}(P)$ denotes the acceptance region for a fixed type of the training sequence in $\mathcal{P}^{m}$. It is interesting to notice that, since in the current setting A has two degrees of freedom, the attack has a twofold effect: the sequence $y^{n}$ is modified in order to bring it inside the acceptance region $\Lambda^{n,*}(P_{t^{m}})$ and the acceptance region itself is modified so to facilitate the former action.

To go on, we find it convenient to rewrite the set $\Gamma^{n}(P_{\tau^{m_{1}}},\lambda,\alpha,L)$ as follows:

[TABLE]

where

[TABLE]

is the set containing all the test sequences (or, equivalently, test types) for which it is possible to corrupt the training set in such a way that they fall within the acceptance region. As the subscript [math] suggests, this set corresponds to the set in (36) when A cannot modify the sequence drawn from $Y$ (i.e. $L=0$) and then tries to hamper the decision by corrupting the training sequence only.

By considering the expression of the acceptance region, the set $\Gamma^{n}_{0}(P_{\tau^{m_{1}}},\lambda,\alpha)$ can be expressed in a more explicit form as follows:

[TABLE]

where the second argument of $h()$ denotes a type in $\mathcal{P}^{m_{1}}$ obtained from the original training sequence $\tau^{m_{1}}$ by first adding $m_{2}$ samples and later removing (in a possibly different way) the same number of samples. Note that in this formulation $Q$ accounts for the fake samples introduced by the attacker and $Q^{\prime}$ for the worst case guess made by the defender of the position of the corrupted samples. We also observe that since we are treating the $SI_{c\textnormal{-}tr}^{a,t}$ game, in general $Q$ will depend on $P_{y^{n}}$. As usual, we implicitly assume that $Q$ and $Q^{\prime}$ are chosen in such a way that $P_{\tau^{m_{1}}}+\frac{\alpha}{(1-\alpha)}(Q-Q^{\prime})$ is nonnegative and smaller than or equal to 1 for all the alphabet symbols.

We are now ready to derive the asymptotic payoff of the game by following a path similar to that used in [2], [3]. First of all we generalise the definition of the sets $\Lambda^{n\times m,*}$, $\Gamma^{n}$ and $\Gamma^{n}_{0}$ so that they can be evaluated for a generic pmf in $\mathcal{P}$ (that is, without requiring that the pmf’s are induced by sequences of finite length). This step passes through the generalization of the $h$ function. Specifically, given any pair of pmf’s $(P,P^{\prime})\in\mathcal{P}\times\mathcal{P}$, we define:

[TABLE]

where $c\in[0,1]$. Note that when $(P,P^{\prime})\in\mathcal{P}^{n}\times\mathcal{P}^{n}$, $h_{c}(P,P^{\prime})=h(P,P^{\prime}).$ The asymptotic version of $\Lambda^{n\times m,*}$ is:

[TABLE]

In a similar way, we can derive the asymptotic versions of $\Gamma^{n}$ and $\Gamma^{n}_{0}$ in (37) and (38)-(39). To do so, we first observe that, the transportation map $S_{YZ}^{n}$ depends on the sources only through the pmfs. By denoting with $S_{PV}^{n}$ a transportation map from a pmf $P\in\mathcal{P}^{n}$ to another pmf $V\in\mathcal{P}^{n}$ and rewriting the set $\Gamma^{n}$ accordingly, we can easily derive the asymptotic version of the set as follows:

[TABLE]

with

[TABLE]

where the definitions of $S_{PV}$ and $\mathcal{A}(L,P)$ derive from those of $S_{PV}^{n}$ and $\mathcal{A}^{n}(L,P)$ by relaxing the requirement that the terms $S_{PV}(i,j)$ and $P(i)$ are rational number with denominator $n$. We now have all the necessary tools to prove the following theorem.

Theorem 2** (Asymptotic payoff of the $SI_{c\textnormal{-}tr}^{a,t}$  game).**

For the $SI_{c\textnormal{-}tr}^{a,t}$  game, the false negative error exponent at the equilibrium is given by

[TABLE]

Accordingly,

if $P_{Y}~{}\in~{}\Gamma(P_{X},\lambda,\alpha,L)$  then  $\varepsilon~{}=~{}0$; 2. 2.

if $P_{Y}~{}\notin~{}\Gamma(P_{X},\lambda,\alpha,L)$  then  $\varepsilon~{}>~{}0$.

Proof.

The theorem could be proven going along the same lines of the proof of Theorem 4 in [3]. We instead provide a proof based on the extension of Sanov’s theorem provided in the Appendix (see Theorem 6). In fact, Theorem 2, as well as Theorem 4 in [3], can be seen as an application of such a generalized version of Sanov’s theorem.

Let us consider

[TABLE]

We start by deriving an upper-bound of the false negative error probability. We can write:

[TABLE]

where the use of the minimum instead of the infimum is justified by the fact that $\Gamma^{n}(R,\lambda,\alpha,L)$ and $\Gamma(R,\lambda,\alpha,L)$ are compact sets. By taking the log and dividing by $n$ we find:

[TABLE]

where $\beta_{n}=|\mathcal{X}|\frac{\log(n+1)((1-\alpha)nc+1)}{n}$ tends to 0 when $n$ tends to infinity.

We now turn to the analysis of a lower bound for $P_{fn}$. Let $R^{*}$ be the pmf achieving the minimum in the outer minimisation of eq. (44). Due to the density of rational numbers within real numbers, we can find a sequence of pmfs’ $R_{m_{1}}\in\mathcal{P}^{m_{1}}$ ($m_{1}=(1-\alpha)nc$) that tends to $R^{*}$ when $n$ (and hence $m_{1}$) tends to infinity. We can write:

[TABLE]

where in the first inequality we have replaced the sum with the single element of the subsequence $R_{m_{1}}$ defined previously, and where the second inequality derives from the well known lower bound on the probability of a type class [8]. From (IV-C), by taking the log and dividing by $n$, we obtain:

[TABLE]

where $\beta_{n}^{\prime}=|\mathcal{X}|\frac{\log(m_{1}+1)}{n}$ tends to 0 when $n$ tends to infinity. In order to compute the probability $P_{Y}(\Gamma^{n}(R_{m_{1}},\lambda,\alpha,L))$, we resort to Corollary 1 of the the generalised version of Sanov’s Theorem given in Appendix -A.

To apply the corollary, we must show that $\Gamma^{n}(R_{m_{1}},\lambda,\alpha,L)\overset{H}{\rightarrow}\Gamma(R^{*},\lambda,\alpha,L)$.

First of all, we observe that by exploiting the continuity of the $h_{c}$ function and the density of rational numbers into the real ones, it is easy to prove that $\Gamma_{0}^{n}(R_{m_{1}},\lambda,\alpha)\overset{H}{\rightarrow}\Gamma_{0}(R^{*},\lambda,\alpha)$. Then the Hausdorff convergence of $\Gamma^{n}(R_{m_{1}},\lambda,\alpha,L)$ to $\Gamma(R^{*},\lambda,\alpha,L)$ follows from the regularity properties of the set of transportation maps stated in Appendix -B. To see how, we observe that any transformation $S_{PV}\in\mathcal{A}(L,P)$ mapping $P$ into $V$ can be applied in inverse order through the transformation $S_{VP}(i,j)=S_{PV}(j,i)$. It is also immediate to see that $S_{VP}$ introduces the same distortion introduced by $S_{PV}$, that is $S_{VP}\in\mathcal{A}(L,V)$. Let now $P$ be a point in $\Gamma(R^{*},\lambda,\alpha,L)$. By definition we can find a map $S_{PV}\in\mathcal{A}(L,P)$ such that $V\in\Gamma_{0}(R^{*},\lambda,\alpha)$. Since $\Gamma_{0}^{n}(R_{m_{1}},\lambda,\alpha)\overset{H}{\rightarrow}\Gamma_{0}(R^{*},\lambda,\alpha)$, for large enough $n$, we can find a point $V^{\prime}\in\Gamma_{0}^{n}(R_{m_{1}},\lambda,\alpha)$ which is arbitrarily close to $V$. Thanks to the second part of Theorem 7 in Appendix -B, we know that a map $S_{V^{\prime}P^{\prime}}\in\mathcal{A}^{n}(L,V^{\prime})$ exists such that $P^{\prime}$ is arbitrarily close to $P$ and $P^{\prime}\in\mathcal{P}^{n}$. By applying the inverse map $S_{P^{\prime}V^{\prime}}$ to $P^{\prime}$, we see that $P^{\prime}\in\Gamma^{n}(R_{m_{1}},\lambda,\alpha,L)$, thus permitting us to conclude that, when $n$ increases, $\delta_{\Gamma(R^{*},\lambda,\alpha,L)}(\Gamma^{n}(R_{m_{1}},\lambda,\alpha,L))\rightarrow 0$. In a similar way, we can prove that $\delta_{\Gamma^{n}(R_{m_{1}},\lambda,\alpha,L)}(\Gamma(R^{*},\lambda,\alpha,L))\rightarrow 0$, hence permitting us to conclude that $\Gamma^{n}(R_{m_{1}},\lambda,\alpha,L)\overset{H}{\rightarrow}\Gamma(R^{*},\lambda,\alpha,L)$.

We can now apply the generalised version of Sanov Theorem as expressed in Corollary 1 of Appendix -A to conclude that:

[TABLE]

Going back to equation (49), and by exploiting the continuity of the divergence function, we can say that for large $n$ we have:

[TABLE]

where the sequence $\nu_{n}$ tends to zero when $n$ tends to infinity. By coupling equations (47) and (51) and by letting $n\tiny\to\infty$, we eventually obtain:

[TABLE]

thus proving the theorem.

∎

As an immediate consequence of Theorem 2, the set $\Gamma(P_{X},\lambda,\alpha,L)$ defines the indistinguishability region of the test, that is the set of all the sources for which A induces D to decide in favour of $H_{0}$ even if $H_{1}$ holds.

IV-D Analysis of the $SI_{c\textnormal{-}tr}^{a}$ game

We now focus on the $SI_{c\textnormal{-}tr}^{a}$ game. For a given choice of $Q(P_{\tau^{m_{1}}})\in\SS_{A,T}$ (and hence $t^{m}$), given a sequence $y^{n}$, the optimum choice of the second part of the attack derives quite easily from the definition of $\Lambda^{n\times m,*}$, namely

[TABLE]

Now the point is to determine the strategy $Q(P_{\tau^{m_{1}}})$ which maximises the probability that the attack in (53) succeeds. To this purpose, of course, the attacker must exploit the knowledge of $P_{Y}$. Since solving such a maximisation problem is not an easy task, we will proceed in a different way. We first introduce a simple (and possibly suboptimum) strategy, then we argue that such a strategy is asymptotically optimum, in that the set of the sources that cannot be distinguished from $X$ with this choice is the same set that we have obtained for the $SI_{c\textnormal{-}tr}^{a,t}$  setup, which is known to be more favourable to the attacker. More specifically, we consider the following two-step attacking strategy. In the first step of the attack, A does not know $y^{n}$, hence he trusts the law of large numbers and optimises $Q(P_{\tau^{m_{1}}})$ by using $P_{Y}$ as a proxy for $P_{y^{n}}$. To do so, he applies equation (32), by replacing $P_{y^{n}}$ with $P_{Y}$. Specifically, by indicating with $Q^{\dagger}$, the resulting strategy for the first step of the attack, we have

[TABLE]

As a by-product of the above minimisation, the attacker also finds the map $S^{n,\dagger}_{YZ}$ representing the optimum attack when $P_{y^{n}}=P_{Y}$. Let us indicate the result of the application of such a map to $P_{Y}$ by $P^{\dagger}_{Z}$.

In the second part of the attack, A tries to move $P_{y^{n}}$ as close as possible to $P^{\dagger}_{Z}$, that is:

[TABLE]

where $S^{n,\dagger}_{YZ}(P_{y^{n}},P_{t^{m}}^{\dagger})$ depends upon the corrupted training sequence obtained after the application of the first part of the attack, namely $P_{t^{m}}^{\dagger}=(1-\alpha)P_{\tau^{m_{1}}}+\alpha Q^{\dagger}(P_{\tau^{m_{1}}})$, through $P_{Z}^{\dagger}$.

The asymptotic optimality of the strategy ($Q^{\dagger}(P_{\tau^{m_{1}}})$, $S^{n,\dagger}_{YZ}(P_{y^{n}},P_{t^{m}}^{\dagger})$) derives from the following theorem

Theorem 3** (Indistinguishability region of the $SI_{c\textnormal{-}tr}^{a}$  game).**

The indistinguishability region of $SI_{c\textnormal{-}tr}^{a}$  game is equal to that of the $SI_{c\textnormal{-}tr}^{a,t}$  game (see eq. (42)) and is asymptotically achieved by the attacking strategy ($Q^{\dagger}(P_{\tau^{m_{1}}})$, $S^{n,\dagger}_{YZ}(P_{y^{n}},P_{t^{m}}^{\dagger})$).

Proof (sketch).

The theorem derives from the observation that due to the law of large numbers, when $n$ grows, $P_{y^{n}}$ tends to $P_{Y}$; hence, for large enough $n$, optimising the first part of the attack by replacing $P_{y^{n}}$ with $P_{Y}$ does not introduce a significant performance loss. The rigorous proof goes along similar lines to those used to prove Theorem 2 and ultimately relies on the continuity of the $h_{c}$ function and the regularity properties of the set $\mathcal{A}^{n}(L,P_{y^{n}})$. The details of the proof are omitted for sake of brevity. ∎

Given that asymptotic equivalence of the $SI_{c\textnormal{-}tr}^{a}$  and the $SI_{c\textnormal{-}tr}^{a,t}$  games, in the rest of the paper, we will generally refer to the $SI_{c\textnormal{-}tr}^{a}$ game without specifying if we are considering the targeted or non-targeted case.

V Source distinguishability for the $SI_{c\textnormal{-}tr}^{a}$ game

In this section, we study the behaviour of the $SI_{c\textnormal{-}tr}^{a}$ game when we vary the decay rate of the false positive error probability $\lambda$. By letting $\lambda$ tend to zero, in fact, we can derive the best achievable performance of the defender when we require only that $P_{fp}$ tends to zero exponentially fast regardless of the decay rate. Then, we use such a result to derive the conditions under which the reliable distinction between two sources is possible in terms of number of corrupted training samples $\alpha$ and maximum allowed distortion $L$.

V-A Ultimate achievable performance of the game

As we said, the goal of this section is to study the limit of the indistinguishability region when $\lambda\tiny\to 0$. This limit, in fact, determines all the pmf’s $P_{Y}$ that can not be distinguished from $P_{X}$ ensuring that the two types of error probabilities tend to zero exponentially fast (with vanishingly small, yet positive, error exponents).

We start by exploiting optimal transport theory to rewrite the indistinguishability region as:

[TABLE]

where EMD (Earth Mover Distance) is the term used in computer vision to denote the minimum transportation cost [19, 20], that is

[TABLE]

With this definition, the main result of this section is stated by the following theorem.

Theorem 4**.**

Given two sources $X$ and $Y$, a maximum allowed average per-letter distortion $L$ and a fraction $\alpha$ of training samples provided by the attacker, the maximum achievable false negative error exponent $\varepsilon$ for the $SI_{c\textnormal{-}tr}^{a}$  game is:

[TABLE]

where $\Gamma(R,\alpha,L)=\Gamma(R,\lambda=0,\alpha,L)$. Accordingly, the ultimate indistinguishability region is given by:

[TABLE]

where $\Gamma_{0}(P_{X},\alpha)=\Gamma_{0}(P_{X},\lambda=0,\alpha)$. Moreover, $\Gamma(P_{X},\alpha,L)$ can be rewritten as:

[TABLE]

with $[a]^{+}=\max\{a,0\}$.

Proof.

The proof of the first part goes along the same steps used in the proof of Theorems 3 and 4 in [4] and is not repeated here. We show, instead, that $\Gamma(P_{X},\alpha,L)$ can be rewritten as in (4).

By observing that $h_{c}(P,Q)=0$ if and only if $P=Q$, it is immediate to see that the set $\Gamma_{0}(P_{X},\lambda=0,\alpha)$ takes the following expression:

[TABLE]

Expression (62) can be rewritten by avoiding the introduction of the auxiliary pmf’s $Q$ and $Q^{\prime}$. To do so, we observe that $Q(i)$ must be larger than $Q^{\prime}(i)$ for all the bins $i$ for which $P(i)>P_{X}(i)$ (and viceversa). In addition, $Q$ and $Q^{\prime}$ must be valid pmf’s, hence we have $\sum_{i}[Q(i)-Q^{\prime}(i)]^{+}=\sum_{i}[Q^{\prime}(i)-Q(i)]^{+}\leq 1$. Then, it is easy to see that (62) is equivalent to the following definition:

[TABLE]

where the second equality follows by observing that $d_{L_{1}}(P,P_{X})=\sum_{i}[P(i)-P_{X}(i)]^{+}+\sum_{i}[P_{X}(i)-P(i)]^{+}$. Eventually, equation (4) derives immediately from the expression of $\Gamma_{0}(P_{X},\alpha)$ given in (63). ∎

According to Theorem 4, $\Gamma(P_{X},\alpha,L)$ provides the ultimate indistinguishability region of the test, that is the set of all the pmf’s for which A wins the game.

Before going on, we pose to discuss the geometrical meaning of the set $\Gamma_{0}(P_{X},\alpha)$ in (62). To do so, we introduce the set $\Lambda_{0}^{*}$, obtained from $\Lambda^{*}$ by letting $\lambda\rightarrow\infty$:

[TABLE]

As usual, we can fix the pmf $P$ and define:

[TABLE]

By referring to Figure 3 (left part), we can geometrically interpret $\Lambda_{0}^{*}(P)$ as the set of the pmf’s $P^{\prime}$ such that $P$ is a convex combination (with coefficient $\alpha$) of $P^{\prime}$ with a point $Q$ of the probability simplex. Starting from (43), we can then rewrite $\Gamma_{0}(P_{X},\alpha)$ as follows:

[TABLE]

Accordingly, $\Gamma_{0}(P_{X},\alpha)$ is geometrically obtained as the union of the acceptance regions built from the points which can be written as a convex combination of $P_{X}$ with some point $Q$ in the simplex. As shown in the right part of Figure 3, such a region corresponds to an hexagon centred in $P_{X}$, which, in the probability simplex, is equivalent to the set of points whose $L_{1}$ distance from $P_{X}$ is smaller than or equal to $2\alpha/(1-\alpha)$ (as stated in (63)). Of course, only the points of the hexagon that lie inside the simplex are valid pmf’s and then must be accounted for.

A pictorial representation of the set $\Gamma(P_{X},\alpha,L)$ is given in Figure 4.

V-B Security margin and blinding corruption level ($\alpha_{b}$)

By a closer inspection of the ultimate indistinguishability region $\Gamma(P_{X},\alpha,L)$, we can derive some interesting parameters characterising the distinguishability of two sources in adversarial setting. Let $X\sim P_{X}$ and $Y\sim P_{Y}$ be two sources. Let us focus first on the case in which the attacker can not modify the test sequence ($L=0$). In this situation, the ultimate indistinguishability region boils down to $\Gamma_{0}(P_{X},\alpha)$. Then we conclude that D can tell the two sources apart if $d_{L_{1}}(P_{Y},P_{X})>\frac{2\alpha}{(1-\alpha)}$. On the contrary, if $d_{L_{1}}(P_{Y},P_{X})\leq\frac{2\alpha}{(1-\alpha)}$, A is able to make the sources indistinguishable by corrupting the training sequence. Clearly, the larger the $\alpha$ the easier is for A to win the game. We can define the blinding corruption level $\alpha_{b}$, as the minimum value of $\alpha$ for which two sources $X$ and $Y$ can not be distinguished. Specifically, we have:

[TABLE]

From (67) it is easy to see that $\alpha_{b}$ is always lower than $1/2$, with the limit case $\alpha_{b}=1/2$ corresponding to a situation in which $P_{X}$ and $P_{Y}$ have completely disjoint supports101010We remind that for any pair of pmf’s $(P,Q)$, $d_{L_{1}}(P,Q)~{}\leq~{}2$.. It is interesting to notice that $\alpha_{b}$ is symmetric with respect to the two sources. Since the attacker is allowed only to add samples to the training sequence without removing existing samples, this might seem a counterintuitive result. Actually, the symmetry of $\alpha_{b}$ is a consequence of the worst case approach adopted by the defender. In fact, D itself discards a subset of samples from the training sequence in such a way to maximise the probability that the remaining part of the training sequence and the test sequence have been drawn from the same source.

Let us now consider the more general case in which $L\neq 0$. For a given $\alpha<\alpha_{b}$, we look for the maximum distortion allowed to $A$ for which it is possible to reliably distinguish between the two sources. From equation (4), we see that the attack does not succeed if:

[TABLE]

This leads to the following definition, which extends the concept of security margin, introduced in [4], to the more general setup considered in this paper.

Definition 3** (Security Margin in the $SI_{c\textnormal{-}tr}^{a}$  setup).**

Let $X\sim P_{X}$ and $Y\sim P_{Y}$ be two discrete memoryless sources. The maximum distortion allowed to the attacker for which the two sources can be reliably distinguished in the $SI_{c\textnormal{-}tr}^{a}$  setup with a fraction $\alpha$ of possibly corrupted samples, is called Security Margin and is given by

[TABLE]

where $L_{\alpha}^{*}=0$ if $P_{Y}\in\Gamma_{0}(P_{X},\alpha)$, while, if $P_{Y}\notin\Gamma_{0}(P_{X},\alpha)$, $L_{\alpha}^{*}$ is the quantity which satisfies

[TABLE]

A geometric interpretation of $L^{*}_{\alpha}$ is given in Figure 5. By focusing on the case $P_{Y}\notin\Gamma_{0}(P_{X},\alpha)$, and by observing that

[TABLE]

is a monotonic non-increasing function of $L$, the security margin can be expressed in explicit form as

[TABLE]

When $L>\SS\mathcal{M}_{\alpha}(P_{X},P_{Y})$, it is not possible for D to distinguish between the two sources with positive error exponents of the two kinds.

By looking at the behavior of the security margin as a function of $\alpha$, we see that $\SS\mathcal{M}_{\alpha_{b}}(P_{X},P_{Y})=0$, meaning that, whenever the fraction of corrupted samples reaches the critical value, the sources can not be distinguished even if the attacker does not introduce any distortion. On the contrary, setting $\alpha=0$ corresponds to study the distinguishability of the sources with uncorrupted training; in this case we have $\SS\mathcal{M}_{0}(P_{X},P_{Y})=\text{\em EMD}(P_{X},P_{Y})$, in agreement with [4]. With reference to Figure 5, it is easy to see that when $\alpha=0$ the hexagon representing $\Gamma_{0}(P_{X},\alpha)$ collapses into the single point $P_{X}$ and the security margin corresponds to the Earth Mover Distance between $Y$ and $X$. Eventually, we notice that, for $\alpha>0$, the value of the security margin in (72) is less than $\text{\em EMD}(P_{X},P_{Y})$. This is also an expected behaviour since the general setting considered in this paper is more favourable to the attacker than the setting in [4].

By looking at (72), we can argue that the Security Margin is symmetric with respect to the two sources $X$ and $Y$, that is, $\SS\mathcal{M}_{\alpha}(P_{Y},P_{X})=\SS\mathcal{M}_{\alpha}(P_{X},P_{Y})$.

To show that this is the case, we observe that the pmf $V^{\prime}$ associated with the minimum $L$, for which we have $\text{\em EMD}(P_{Y},V^{\prime})=\SS\mathcal{M}_{\alpha}(P_{X},P_{Y})$, can be obtained through the application of a map $S_{P_{Y}V}$ that works as follows: it does not modify a portion $\alpha/(1-\alpha)$ of $P_{Y}$ and moves the remaining mass into an equal amount of $P_{X}$ in a convenient way (i.e., in such a way to minimise the overall distance between the masses). The inverse map can be applied to bring the same quantity of mass from $P_{X}$ to $P_{Y}$, while leaving as is the remaining mass, thus obtaining a $V^{\prime\prime}$ which satisfies $\text{\em EMD}(P_{X},V^{\prime\prime})=\text{\em EMD}(P_{Y},V^{\prime})$ (because of the symmetry of the per-symbol distortion $d$) and $d_{L_{1}}(V^{\prime\prime},P_{Y})=d_{L_{1}}(V^{\prime},P_{X})=2\alpha/(1-\alpha)$. Arguably, $V^{\prime\prime}$ is the pmf for which $\text{\em EMD}(P_{X},V^{\prime\prime})=\SS\mathcal{M}_{\alpha}(P_{Y},P_{X})$; hence, $\SS\mathcal{M}_{\alpha}(P_{Y},P_{X})=\SS\mathcal{M}_{\alpha}(P_{X},P_{Y})$.

V-B1 Bernoulli sources

In order to get some insights on the practical meaning of $\alpha_{b}$ and $\SS\mathcal{M}_{\alpha}$, we consider the simple case of two Bernoulli sources with parameter $q=P_{X}(1)$ and $p=P_{Y}(1)$. Assuming that no distortion is allowed to the attacker, the minimum fraction of samples that A must add to induce a decision error is, according to (67), $\alpha_{b}=\frac{|p-q|}{1+|p-q|}$. For instance, and rather obviously, when $|p-q|=1$, to win the game A must introduce a number of fake samples equal to the number of samples of the correct training sequence, i.e. $\alpha=0.5$. With regard to $\SS\mathcal{M}$, we have:

[TABLE]

Figure 6 illustrates the behavior of $\SS\mathcal{M}_{\alpha}(p,q)$ as a function of $\alpha$ when $p=0.3$ and $q=0.7$. The blinding corruption value is $\alpha_{b}=0.286$.

VI Source identification game with replacement of training samples

In this section, we study a variant of the game with corrupted training, in which A observes the training sequence and can replace a selected fraction of samples. Let $\tau^{m}$ indicate the original $m$-sample long training sequence drawn from $X$ and let $\mathcal{M}$ be a subset of $m_{2}~{}=~{}\alpha m$ indexes in $[1,2\dots m]$. The attacker can choose the index set $\mathcal{M}$ and replace the corresponding samples with $m_{2}$ fake samples. More formally, given the original training sequence $\tau^{m}$, the training sequence seen by the defender is t^{m}=\sigma(\tau_{\bar{{\scalebox{0.8}{\scriptscriptstyle\mathcal{M}}}}}^{m_{1}}||\tau^{m_{2}}), where $\bar{\mathcal{M}}$ is the complement of $\mathcal{M}$ in $[1,2\dots m]$, \tau_{\bar{{\scalebox{0.8}{\scriptscriptstyle\mathcal{M}}}}}^{m_{1}} is the set of original (non-attacked) samples, and $\tau^{m_{2}}$ is the sequence with the fake samples introduced by the attacker.

Figure 7 illustrates the adversarial setup considered in this section for the case of a targeted attack. Arguably, this scenario is more favourable to the attacker with respect to the $SI_{c\textnormal{-}tr}^{a}$ game.

VI-A Formal definition of the $SI_{c\textnormal{-}tr}^{r}$ game

In the sequel, we formally define the source identification game with replacement of selected samples, namely the $SI_{c\textnormal{-}tr}^{r}$ game. As anticipated, we focus on a version of the game in which the corruption of the training samples depends on the to-be-attacked sequence $y^{n}$ (targeted attack), the extension to the case of non-target attack, in fact, can be easily obtained by following the same approach used in Section IV-D.

VI-A1 Defender’s strategies

As in the $SI_{c\textnormal{-}tr}^{a}$ game, in order to be sure that the false positive error probability is lower than $2^{-n\lambda}$, the defender adopts a worst case strategy and considers the maximum of the false positive error probability over all the possible $P_{X}$ and over all the possible attacks that the training sequence may have undergone, yielding:

[TABLE]

While the above expression is formally equal to that of the $SI_{c\textnormal{-}tr}^{a}$ game (see eq. (15)), the maximisation over $\SS_{A,T}$ is now more cumbersome, due to the additional degree of freedom available to the attacker, who can selectively remove the samples of the original training sequence. In fact, even if D knew the position of the corrupted samples, simply throwing them away would not guarantee that the remaining part of the sequence would follow the same statistics of $X$, since the attacker might have deliberately altered them by selectively choosing the samples to replace.

VI-A2 Attacker’s strategies

With regard to the attacker, the part of the attack working on the test sequence $y^{n}$ is the same as for the $SI_{c\textnormal{-}tr}^{a}$ case, while the part regarding the corruption of the training sequence must be redefined. To this purpose, we observe that the corrupted training sequence may be any sequence $t^{m}$ for which $d_{H}(t^{m},\tau^{m})\leq\alpha m$, where $d_{H}$ denotes the Hamming distance. Given that the defender basis his decision on the type of $t^{m}$, it is convenient to rewrite the constraint on the Hamming distance between sequences as a constraint on the $L_{1}$ distance between the corresponding types. In fact, by looking at the empirical distribution of the corrupted sequence, searching for a sequence $t^{m}$ s.t. $d_{H}(t^{m},\tau^{m})\leq\alpha m$ is equivalent to searching for a pmf $P_{t^{m}}\in\mathcal{P}^{m}$ for which $d_{L_{1}}(P_{t^{m}},P_{\tau^{m}})\leq 2\alpha$ (see the proof of Lemma 2 in [2]). Therefore, the set of strategies of the attacker is defined by $\SS_{A}=\SS_{A,T}\times\SS_{A,O}$, where

[TABLE]

Note that, in this case, the function $Q(\cdot,\cdot)$ gives the type of the whole training sequence observed by D (not only the fake subpart, as it was in the $SI_{c\textnormal{-}tr}^{a}$  game), that is, $P_{t^{m}}=Q(P_{\tau^{m}},P_{y^{n}})$.

In the following, we will find convenient to express the attacking strategies in $\SS_{A,T}$ in an alternative way. Since the attacker replaces the samples of a subpart of the training sequence, the corruption strategy is equivalent to first removing a subpart of the training sequence and then adding a fake subsequence of the same length. Then, the sequence is reordered to hide the position of the fake samples. By focusing on the type of the observed training sequence, we can write:

[TABLE]

where $Q_{R}(P_{\tau^{m}},P_{y^{n}})$ and $Q_{A}(P_{\tau^{m}},P_{y^{n}})$ (both belonging to $\mathcal{P}^{m_{2}}$) are the types of the removed and injected subsequences respectively. In order to simplify the notation, in the following we will avoid to indicate explicitly the dependence of $Q_{R}(P_{\tau^{m}},P_{y^{n}})$ and $Q_{A}(P_{\tau^{m}},P_{y^{n}})$ on $P_{\tau^{m}}$, $P_{y^{n}}$, and will indicate them as $Q_{R}()$ and $Q_{A}()$. Furthermore, we will use notation $Q_{R}$ and $Q_{A}$ whenever the dependence from the arguments is not relevant. By varying $Q_{R}$ and $Q_{A}$, we obtain all the pmf’s that can be produced from $P_{{\tau}^{m}}$ by first removing and later adding $m_{2}$ samples. Of course not all pairs $(Q_{R},~{}Q_{A})$ are admissible since the $P_{t^{m}}$ resulting from eq. (77) must be a valid pmf, i.e. it must be nonnegative for all the symbols of the alphabet $\mathcal{X}$.

VI-A3 Payoff

As usual, the payoff function is defined as

[TABLE]

VI-B Equilibrium point and payoff at the equilibrium

In order to ensure that $P_{fp}$ is always lower than $2^{-\lambda n}$ , it is convenient to use the attack formulation given in (77). For a given $P_{X}$, $Q_{R}$ and $Q_{A}$, $P_{fp}$ is the probability that $X$ generates two sequences $x^{n}$ and $\tau^{m}$, such that the pair of type classes $(P_{x^{n}},P_{\tau^{m}}-\alpha(Q_{R}()-Q_{A}()))$ falls outside $\Lambda^{n\times m}$. Accordingly, the set of strategies available to D can be rewritten as:

[TABLE]

By proceeding as in the proof of Lemma 1, it is easy to prove that the asymptotically optimum strategy for the defender corresponds to the following:

[TABLE]

where $\delta_{n}$ tends to 0 as $n\rightarrow\infty$ and the minimization is limited to $Q_{R}$ and $Q_{A}$ in $\mathcal{P}^{m_{2}}$ such that $P_{t^{m}}+\alpha(Q_{R}-Q_{A})$ is a valid pmf. Consequently, the optimum attacking strategy is given by:

[TABLE]

hence resulting in the following theorem.

Theorem 5**.**

The $SI_{c\textnormal{-}tr}^{r}$  game with targeted corruption is a dominance solvable game, whose only rationalizable equilibrium corresponds to the profile $(\Lambda^{n\times m,*},(Q^{*}(),~{}S^{n,*}_{YZ}()))$ given by equations (VI-B) and (81).

In order to study the asymptotic payoff of the $SI_{c\textnormal{-}tr}^{r}$  game at the equilibrium, we parallel the analysis carried out in Sec. IV-C. By considering the case $L=0$, the set of pairs of types for which D will accept $H_{0}$ as a consequence of the attack to the training sequence is given by

[TABLE]

If we fix the type of the original training sequence, we get:

[TABLE]

By letting $n$ go to infinity, we obtain the asymptotic counterpart of the above set, which, for a generic $R\in\mathcal{P}$, takes the following expression:

[TABLE]

When $L\neq 0$, we obtain:

[TABLE]

With the above definitions, it is straightforward to extend Theorem 2 to the $SI_{c\textnormal{-}tr}^{r}$  case, thus proving that the set in (85) evaluated in $R~{}=~{}P_{X}$ represents the indistinguishability region of the $SI_{c\textnormal{-}tr}^{r}$  game.

VI-C Security margin and blinding corruption level

As a last contribution, we are interested in studying the ultimate distinguishability of two sources $X$ and $Y$ in the $SI_{c\textnormal{-}tr}^{r}$  setting and compare it with the result we have obtained for the $SI_{c\textnormal{-}tr}^{a}$  case. To do so, we consider the behaviour of the indistinguishability region when $\lambda$ tends to 0. We have:

[TABLE]

where

[TABLE]

The set in (87) can be equivalently rewritten as

[TABLE]

To see why, we first notice that set (87) is contained in (88). Indeed, from the triangular inequality we have that, for any $P^{\prime}$, $d(P,P_{X})~{}\leq~{}d_{L_{1}}(P,P^{\prime})+d_{L_{1}}(P^{\prime},P_{X})$. Then, if $P$ belongs to $\Gamma_{0}(P_{X},\alpha)$ in (87), it also belongs to the set in (88). To see that the two sets are indeed equivalent, it is sufficient to show that the reverse implication also holds. To this purpose, we observe that, whenever $d_{L_{1}}(P,P_{X})~{}\leq~{}4\alpha$, a type $P^{*}$ can be found such that its distance from both $P$ and $P_{X}$ is less or at most equal to $2\alpha$. In fact, by letting $P^{*}~{}=~{}\frac{P+P_{X}}{2}$, we have

[TABLE]

If $d_{L_{1}}(P,P_{X})~{}\leq~{}4\alpha$, then, $d_{L_{1}}(P,P^{*})~{}=~{}d_{L_{1}}(P^{*},P_{X})$ $~{}=~{}d_{L_{1}}(P,P_{X})/2~{}\leq~{}2\alpha$, permitting us to conclude that the sets in (87) and (88) are equivalent.

Upon inspection of equation (88), we can conclude that, as expected, the indistinguishability region for $L=0$ (and hence, also for the case $L\neq 0$) is larger than that of the $SI_{c\textnormal{-}tr}^{a}$  game (see (63)), thus confirming that the game with sample replacement is more favourable to the attacker (a graphical comparison between the indistinguishability regions for the two setups is shown in Figure 8).

As a matter of fact, for the attacker, the advantage of the $SI_{c\textnormal{-}tr}^{r}$  game with respect to the $SI_{c\textnormal{-}tr}^{a}$  game depends on $\alpha$. For small $\alpha$ and for $\alpha$ close to $1/2$, the indistinguishability regions of the two games are very similar, while for intermediate values of $\alpha$ the indistinguishability region of the $SI_{c\textnormal{-}tr}^{r}$  game is considerably larger than that of the $SI_{c\textnormal{-}tr}^{a}$  game (the maximum difference between the two regions is obtained for $\alpha\approx 0.3$). When $\alpha=1/2$ the attacker always wins, since he is able to bring any pmf inside the acceptance region regardless of the game version, while for $\alpha=0$, we fall back into the source identification game without corruption of the training sequence, thus making the two versions of the game equivalent.

Given two sources $X$ and $Y$, the blinding corruption level value takes the expression:

[TABLE]

Since $d_{L_{1}}(P_{Y},P_{X})\leq 2$ for any couple $(P_{Y},P_{X})$ (the maximum value 2 is taken when the two distribution have disjoint support), the blinding value for the $SI_{c\textnormal{-}tr}^{r}$  game is lower than the blinding value of $SI_{c\textnormal{-}tr}^{a}$  game. The two expressions are identical when the two sources have disjoint support, in which case $\alpha_{b}=1/2$.

When the attacker can also corrupt the test sequence, the ultimate indistinguishability region of the $SI_{c\textnormal{-}tr}^{r}$game is:

[TABLE]

Starting from (91) we can define the security margin in the $SI_{c\textnormal{-}tr}^{r}$  setup.

Definition 4** (Security Margin in the $SI_{c\textnormal{-}tr}^{r}$  setup).**

Let $X\sim P_{X}$ and $Y\sim P_{Y}$ be two discrete memoryless sources. The maximum distortion for which the two sources can be reliably distinguished in the $SI_{c\textnormal{-}tr}^{r}$  setup is called Security Margin and is given by

[TABLE]

where $L_{\alpha}^{*}$ is the quantity which satisfies the following relation

[TABLE]

*if $P_{Y}\notin\Gamma_{0}(P_{X},\alpha)$, and $L_{\alpha}^{*}=0$ otherwise. *

Considering again the case of two Bernoulli sources and by adopting the same notation of Section V-B1, we have that $\alpha_{b}=|p-q|/4$, while the security margin is

[TABLE]

Figure 6 plots $\SS\mathcal{M}_{\alpha}$ as a function of $\alpha$ when $p=0.3$ and $q=0.7$. The blinding value is $\alpha_{b}=0.1$ which, as expected, is lower than the value we found for the $SI_{c\textnormal{-}tr}^{a}$  setup.

VII Conclusions

We studied the distinguishability of two sources in an adversarial setup when the sources are known through training data, part of which can be corrupted by the attacker himself. We considered two different scenarios. In the first one, the attacker simply adds fake samples to the original training sequence, while in the second one, the attacker replaces a selected subset of training samples with fake ones. We formalised both cases in a game-theoretic setup, then we derived the equilibrium point of the games and analysed the (asymptotic) payoff at the equilibrium. The result of the game can be summarised in a compact and elegant way by introducing two parameters, namely the Security Margin under corruption of the training sequence, and the blinding corruption level $\alpha_{b}$, defined as the portion of fake samples the attacker must introduce to make impossible any reliable distinction between the sources. Based on these two parameters, the performance of the two games with corruption of the training data can be easily compared.

Though rather theoretical, our findings can guide more practical researches in several fields belonging to the emerging areas of adversarial signal processing [1] and secure machine learning [6]. In many cases, in fact, the defender must take into account the possibility that the data he is using to tune the system he is working at, or during the learning phase, is corrupted by the attacker.

The analysis carried out in this paper can be extended in several ways, for instance by considering continuous sources, or by assuming that the sources $X$ and $Y$ are not memoryless, but still amenable to be studied by using the method of types [21]. Following the analysis in [22], we could also consider a more general setup in which the attacker is active under both $H_{0}$ and $H_{1}$. An interesting generalisation, consists in studying a symmetric setup in which the training and the test sequences can be corrupted by applying the same kinds of processing. For instance, the attacker could be allowed to replace samples in both the training and the set sequences, or he could be allowed to modify the training sequence up to a certain distortion. Other kinds of attacks to the training data could also be considered, like sample removal with no addition of fake samples. As a matter of fact, the kind of attack strongly depends on the application scenario, and it is arguable that the availability of a large variety of theoretical models would help bridging the gap between theory and practice.

Acknowledgment

This work has been partially supported by a research sponsored by DARPA and Air Force Research Laboratory (AFRL) under agreement number FA8750-16-2-0173. The U.S. Government is authorised to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of DARPA and Air Force Research Laboratory (AFRL) or the U.S. Government.

-A Generalized Sanov’s theorem

Let us consider a sequence of $n$ i.i.d. discrete random variables taking values in a finite alphabet $\mathcal{X}$ and distributed according to a pmf $P$. We denote with $P_{n}$ the empirical pmf of the sequence. Let $E\subseteq\mathcal{P}$ be a set of pmf’s. Sanov’s theorem [8, 23, 9] states that

[TABLE]

where int S denote the interior part of the set $S$.

When cl($E$) = cl(int($E$))111111cl($E$) denotes the closure of $E$. Clearly, cl($E$) $\equiv$ $E$ if $E$ is a closed set., or, $E~{}\subseteq~{}$cl(int($E$)), the left and right-hand side of (A1) coincide and we get the exact rate:

[TABLE]

If we define the set $E_{n}=E\cap\mathcal{P}^{n}$, we have: $P(P_{n}\in E)=P(P_{n}\in E_{n})$ and we can rewrite Sanov’s theorem as:

[TABLE]

Note that, by construction, we have cl($E$) = cl($\cup_{n}E_{n}$).

In the following, we extend the formulation of Sanov’s theorem given in (A3) to more general sequences of sets $E_{n}$ for which it does not necessary hold that $E_{n}=E\cap\mathcal{P}^{n}$ for some set $E$.

We start by introducing the notion of convergence for sequences of subsets due to Kuratowsky, which is a more general notion of convergence with respect to the one based on Hausdorff distance. Let $(S,d)$ be a metric space. We first provide the definition of lower closed limit or Kuratowski limit inferior [24].

Definition 5**.**

A point $p$ belongs to the lower limit $\underset{n\rightarrow\infty}{Li}K_{n}$ (or simply ${Li}K_{n}$) of a sequence of sets $K_{n}$, if every neighborhood of $p$ intersects all the $K_{n}$’s from a sufficiently great index $n$ onward.

Given the above definition, the expression $p~{}\in\underset{n\rightarrow\infty}{Li}K_{n}$ is equivalent to the existence of a sequence of points $\{p_{n}\}$ such that:

[TABLE]

Stated in another way, ${Li}K_{n}$ is the set of the accumulation points of sequences in $K_{n}$. As an alternative, equivalent, definition we can let:

[TABLE]

Similarly, we have the following definition of upper closed limit or Kuratowski limit superior [24].

Definition 6**.**

A point $p$ belongs to the upper limit $\underset{n\rightarrow\infty}{Ls}K_{n}$ (or simply ${Ls}K_{n}$) of a sequence of sets $K_{n}$, if every neighborhood of $p$ intersects an infinite number of terms in $K_{n}$.

The expression $p~{}\in~{}{Ls}_{n\rightarrow\infty}K_{n}$ is equivalent to the existence of a subsequence of points $\{p_{k_{n}}\}$ such that

[TABLE]

As an alternative, equivalent, definition we can let:

[TABLE]

It can be proven that the Kuratowski limit inferior and superior are always closed set (see [24]).

Given the above, we can state the following:

Definition 7**.**

The sequence of sets $\{K_{n}\}$ is said to be convergent to $K$ in the sense of Kuratowski, that is $K_{n}\overset{K}{\rightarrow}~{}K$, if $LiK_{n}=K=LsK_{n}$, in which case we write $K=LimK_{n}$.

We observe that Kuratowski convergence is weaker than convergence in Hausdorff metric; in fact, given a sequence of closed sets $\{K_{n}\}$, $K_{n}\overset{H}{\rightarrow}~{}K$ implies $K_{n}\overset{K}{\rightarrow}~{}K$ [25]. For compact metric spaces, the reverse implication also holds and the two kinds of convergence coincide.

In this work, we are interested in the space $\mathcal{P}$ of probability mass functions defined over a finite alphabet $\mathcal{X}$, i.e., the probability simplex in $\mathbb{R}^{|\mathcal{X}|}$, equipped with the $L_{1}$ metric. Being $\mathcal{P}$ a closed subset of $\mathbb{R}^{|\mathcal{X}|}$, $\mathcal{P}$ is a complete set. In addition, with the $L_{1}$ metric, $\mathcal{P}\in\mathcal{L}(\mathbb{R}^{|\mathcal{X}|})$, that is, $\mathcal{P}$ is bounded. The space $(\mathcal{P},d_{L_{1}})$, then, is a compact metric space and then, for our purposes, Kuratowski and Hausdorff convergence are equivalent.

We are now ready to prove the following generalisation of Sanov’s theorem:

Theorem 6** (Generalized Sanov’s theorem).**

*Let $\{E_{(n)}\}$ be a sequence of sets in $\mathcal{P}$, such that $Li(E_{(n)}\cap\mathcal{P}^{n})\neq~{}\emptyset$. Then: *

[TABLE]

If, in addition, $LsE_{(n)}=Li(E_{(n)}\cap\mathcal{P}^{n})$, the generalized Sanov’s limit exists as follows:

[TABLE]

Proof.

We first prove the expression for the lower bound. Let $E_{n}~{}=~{}E_{(n)}\cap\mathcal{P}^{n}$. We have:

[TABLE]

In the last inequality we exploited the fact that, being each $E_{(n)}$ a bounded set of $\mathcal{P}$, and $\mathcal{D}$ lower bounded in $\mathcal{P}$, the infimum over $E_{(n)}$ corresponds to the minimum over its closure. By taking the logarithm of each side and dividing by $n$, we get:

[TABLE]

We now prove that, for any $\delta$ and for sufficiently large $n$, we have

[TABLE]

First, according to the properties of the limit superior, $LsE_{(n)}=Ls(\text{\em cl}(E_{(n)}))$ [24], hence proving (A11) is equivalent to showing that:

[TABLE]

Let $Q_{n}$ be the sequence of points achieving the minimum of the left-hand side of (A12) (for simplicity we assume that the minimum is unique, the extension to a more general case being straightforward). Let $Q_{n(j)}$ be a subsequence of $Q_{n}$ formed only by the elements of $Q_{n}$ that do not belong to $Ls(\text{\em cl}(E_{(n)}))$121212 $n(i)~{}>~{}n(j),\forall i~{}>~{}j$. If the number of elements in $Q_{n(j)}$ is finite, then for $n$ large enough $Q_{n}~{}\in~{}Ls(\text{\em cl}(E_{(n)}))$ and eq. (A12) is verified with $\delta=0$. If the number of elements in $Q_{n(j)}$ is infinite, then, due to the boundedness of $\mathcal{P}$, the elements of $Q_{n(j)}$ must have at least one accumulation point (Bolzano-Weierstrass theorem). Let $A_{i}$’s be the accumulation points of $Q_{n(j)}$. By definition of $Ls$, all $A_{i}$’s belong to $Ls(\text{\em cl}(E_{(n)}))$. In addition, for any radius $\rho$, from a certain $j$ on, all the points in $Q_{n(j)}$ belong to $\mathcal{R}=\bigcup_{i}\mathcal{B}(A_{i},\rho)$131313$\mathcal{B}(A_{i},\rho)$ is a ball with radius $\rho$ centred in $A_{i}$.. For large enough $n$, then we have:

[TABLE]

where the second inequality derives from the continuity of the $\mathcal{D}$ function and the arbitrariness of $\rho$.

By inserting equation (A11) in (A10), we have that, for large $n$,

[TABLE]

and hence, by the arbitrariness of $\delta$,

[TABLE]

We now pass to the upper bound. Let $Q^{*}$ be a point achieving the minimum of the divergence over the set $LiE_{n}$. By definition of limit inferior, there exists a sequence of points $\{Q_{n}\}$, $Q_{n}\in E_{n}$ such that $Q_{n}\rightarrow Q^{*}$ as $n\rightarrow\infty$. Then, by exploiting the continuity of $\mathcal{D}$, it follows that:

[TABLE]

where $\gamma$ can be made arbitrarily small for large $n$. We can then write:

[TABLE]

Hence, we get

[TABLE]

and then, by the arbitrariness of $\gamma$,

[TABLE]

which concludes the proof of the first part (relation (A7)).

For the proof of the second part, we observe that, when $LsE_{(n)}=Li(E_{(n)}\cap\mathcal{P}^{n})$, the two bounds in (A7) coincides. Moreover, the following chain of inclusions holds, $LiE_{(n)}~{}\subseteq~{}LsE_{(n)}~{}=~{}Li(E_{(n)}\cap\mathcal{P}^{n})~{}\subseteq~{}LiE_{(n)}$, and then $LiE_{(n)}~{}=~{}LsE_{(n)}~{}=~{}LimE_{(n)}$, yielding (A8). ∎

We observe that, in general, the Kuratowski convergence of $E_{(n)}$ is a necessary condition for the existence of the generalized Sanov limit in (A8), but it is not sufficient. In fact, we could have $LiE_{(n)}~{}\supseteq~{}Li(E_{(n)}\cap\mathcal{P}^{n})$, in which case the lower and upper bound in (A7) do not coincide. It is also interesting to notice that when $E_{(n)}~{}\in~{}\mathcal{P}^{n}$ is a sequence of sets in $\mathcal{P}^{n}$, then Sanov’s limit holds whenever $E_{(n)}\overset{K}{\rightarrow}E$ for some set $E$, or, by exploiting the compactness of $\mathcal{P}$, $E_{(n)}\overset{H}{\rightarrow}E$. Based on the above observation, we can state the following corollary:

Corollary 1**.**

Let $E_{(n)}$ be a sequence of sets in $\mathcal{P}^{n}$, such that $E_{(n)}\overset{H}{\rightarrow}E$. Then:

[TABLE]

-B Regularity properties of the set of admissible maps

To prove the theorems on the asymptotic behaviour of the payoff in the two versions of the source identification game studied in this paper, we need to prove some regularity theorems on the set of admissible maps.

To start with, we need to define a distance between transportation maps, that is a function $d_{s}:~{}\mathbb{R}^{|\mathcal{X}|\times|\mathcal{X}|}\times\mathbb{R}^{|\mathcal{X}|\times|\mathcal{X}|}\rightarrow\mathbb{R}^{+}$. In accordance with the rest of the paper, let us choose the $L_{1}$ distance, that is, given two maps ($S_{PV},S_{QR}$), we define $d_{s}(S_{PV},S_{QR})=\sum_{i,j}|S_{PV}(i,j)-S_{QR}(i,j)|$.

Our first result regards the regularity of $\mathcal{A}(L,P)$ as a function of $P$.

Lemma 2**.**

Let $P\in\mathcal{P}$ and let $P^{\prime}$ be any pmf in the neighbourhood of $P$ of radius $\tau$, i.e., $P^{\prime}\in\mathcal{B}(P,\tau)$. Then

[TABLE]

and hence $\underset{\tau\rightarrow 0}{\lim}\delta_{H}(\mathcal{A}(L,P),\mathcal{A}(L,P^{\prime}))~{}=~{}0$, uniformly in $\mathcal{P}$.

Moreover, if we insist that $P^{\prime}\in\mathcal{P}^{n}$, the following result holds: $\forall\varepsilon>0,~{}\exists\tau^{*}$ and $n^{*}$ such that $\forall\tau<\tau^{*}$ and $n>n^{*}$,

[TABLE]

Proof.

From a general perspective, the lemma follows from the fact that $\mathcal{A}^{n}(L,P_{y^{n}})$ (and $\mathcal{A}(L,P)$) is built by imposing a number of linear constraints on the admissible transportation maps (see eq. (11)), i.e. $\mathcal{A}(L,P)$ is a convex polytope [26, 27]. By considering a $P^{\prime}$ close to $P$, we are perturbing the vector of the known terms of the linear constraints which defines the admissibility set. Instead of invoking the above general principle, in the following we give an explicit proof of the lemma.

Given $P\in\mathcal{P}$ and $P^{\prime}\in\mathcal{B}(P,\tau)$, let $\tau(i)=P(i)-P^{\prime}(i)$ be the excess (or defect) of mass of $P$ with respect to $P^{\prime}$ in bin $i$. For any map in $\mathcal{A}(L,P)$, we can choose a map $S_{P^{\prime}V^{\prime}}$ that works as follows: for the bins $i$ such that $\tau(i)\leq 0$, let $S_{P^{\prime}V^{\prime}}(i,j)=S_{PV}(i,j)$ for $j\neq i$, while for $j=i$, we let $S_{P^{\prime}V^{\prime}}(i,j)=S_{PV}(i,j)+|\tau(i)|$. For the bins $i$ for which $\tau(i)>0$, we first sort the index set $\{j:S_{PV}(i,j)\neq 0\}$ in decreasing order with respect to the amount of distortion introduced per unit of mass delivered from $i$ to $j$ ($d(i,j)$). Then, starting from the first index in the ordered list, we let $S_{P^{\prime}V^{\prime}}(i,j)=\max(0,~{}S_{PV}(i,j)-\tau(i))$. If $S_{P^{\prime}V^{\prime}}(i,j)=0$, we update $\tau(i)$ to a new value $\tau^{\prime}(i)=\tau(i)-S_{PV}(i,j)$, and iterate the previous procedure by subtracting the updated value of $\tau^{\prime}(i)$ from the second $S_{PV}(i,j)$ in the list. This procedure goes on until the subtraction gives $S_{P^{\prime}V^{\prime}}(i,j)\neq 0$, that is when we have removed all the excess mass from the $i$-th row of $S_{PV}(i,j)$.

It is easy to see that the map built in this way satisfies the distortion constraint, in fact, by construction the distortion associated to $S_{P^{\prime}V^{\prime}}$ is less than that introduced by $S_{PV}$. Then, $S_{P^{\prime}V^{\prime}}\in\mathcal{A}(L,P^{\prime})$. In addition, by construction, $\sum_{j}|S_{P^{\prime}V^{\prime}}(i,j)-S_{PV}(i,j)|\leq|\tau(i)|$, and hence $\sum_{ij}|S_{P^{\prime}V^{\prime}}(i,j)-S_{PV}(i,j)|\leq\tau$. Accordingly, we have:

[TABLE]

since, as we have shown with the preceding construction, the inner minimum is always lower or equal than $\tau$. By repeating the same argument exchanging the role of $\mathcal{A}(L,P)$ and $\mathcal{A}(L,P^{\prime})$, we find that $\delta_{H}(\mathcal{A}(L,P^{\prime}),\mathcal{A}(L,P))\leq\tau$, thus concluding the first part of the proof.

In the second part of the lemma, we require that $P^{\prime}\in\mathcal{P}^{n}$ and that the map produces a sequence in $\mathcal{P}^{n}$. The proof is easily achieved by exploiting the first part of the lemma according to which for any map $S_{PV}$ in $\mathcal{A}(L,P)$, we can find a map $S_{P^{\prime}V^{\prime}}$ in $\mathcal{A}(L,P^{\prime})$ which is arbitrarily close to $S_{PV}$, and then approximating $S_{P^{\prime}V^{\prime}}$ with a map $S^{n}_{P^{\prime}V^{\prime}}\in\mathcal{A}^{n}(L,P^{\prime})$. Due to the density of rational numbers in real numbers, such an approximation can be made arbitrarily accurate by increasing $n$, thus completing the proof. ∎

Given a transformation $S_{PV}$ mapping $P$ into $V$, Lemma 2 states that, for any pmf $P^{\prime}$ close to $P$, we can find a map $S_{P^{\prime}V^{\prime}}$ close to $S_{PV}$. The following theorem extends such a result to the pmf resulting from the application of the mapping.

Theorem 7**.**

Let $P\in\mathcal{P}$, and let $P^{\prime}$ be any pmf in the neighbourhood of $P$ of radius $\tau$, i.e., $P^{\prime}\in\mathcal{B}(P,\tau)$. Let $S_{PV}\in\mathcal{A}(L,P)$. Then, we can always find a map $S_{P^{\prime}V^{\prime}}\in\mathcal{A}(L,P^{\prime})$ such that $V^{\prime}\in\mathcal{B}(V,\tau)$.

Similarly, for any $\varepsilon>0$, there exist $\tau^{*}$ and $n^{*}$ such that $\forall$ $\tau<\tau^{*}$ and $n>n^{*}$, given a $P\in\mathcal{P}$, a map $S_{PV}\in\mathcal{A}(L,P)$ and $P^{\prime}\in\mathcal{P}^{n}\cap\mathcal{B}(P,\tau)$, we can find a map $S_{P^{\prime}V^{\prime}}^{n}$ in $\mathcal{A}^{n}(L,P^{\prime})$ such that $V^{\prime}_{n}\in\mathcal{B}(V,\varepsilon)\cap\mathcal{P}^{n}$.

Proof.

For any two maps $S_{PV}$ and $S_{P^{\prime}V^{\prime}}$, we have:

[TABLE]

and

[TABLE]

yielding:

[TABLE]

By summing over $j$ and exploiting Lemma 2, we can choose $S_{P^{\prime}V^{\prime}}$ so that:

[TABLE]

and hence $V^{\prime}\in\mathcal{B}(V,|\tau)$.

Similarly to the second part of Lemma 2, the second part of the theorem follows immediately from the density of rational numbers in the real line.

∎