Distributed Hypothesis Testing: Cooperation and Concurrent Detection

TL;DR

This paper analyzes distributed hypothesis testing with cooperation between detectors and rate-limited communication, characterizing error exponents and the benefits of cooperation in different communication rate regimes.

Contribution

It provides exact characterizations of error exponents in zero-rate and positive-rate communication settings, highlighting the impact of cooperation on detection performance.

Findings

Exact exponents characterized for zero-rate links.

Cooperation can significantly improve error exponents.

Tradeoffs between detectors' error exponents depend on communication rates.

Abstract

A single-sensor two-detectors system is considered where the sensor communicates with both detectors and Detector 1 communicates with Detector 2, all over noise-free rate-limited links. The sensor and both detectors observe discrete memoryless source sequences whose joint probability mass function depends on a binary hypothesis. The goal at each detector is to guess the binary hypothesis in a way that, for increasing observation lengths, the probability of error under one of the hypotheses decays to zero with largest possible exponential decay, whereas the probability of error under the other hypothesis can decay to zero arbitrarily slow. For the setting with zero-rate communication on both links, we exactly characterize the set of possible exponents and the gain brought up by cooperation, in function of the number of bits that are sent over the two links. Notice that, for this setting, tradeoffs between the exponents achieved at the two detectors arise only in few particular cases. In all other cases, each detector achieves the same performance as if it were the only detector in the system. For the setting with positive communication rates from the sensor to the detectors, we characterize the set of all possible exponents in a special case of testing against independence. In this case the cooperation link allows Detector~2 to increase its Type-II error exponent by an amount that is equal to the exponent attained at Detector 1. We also provide a general inner bound on the set of achievable error exponents. For most cases it shows a tradeoff between the two exponents.

Equations

$$\mathcal{H}=0:\{(X_t,Y_{1,t},Y_{2,t}){\}}_{t=1}^n\text{ i.i.d. }{P}_{X{Y}_1{Y}_2}$$

$$\mathcal{H}=1:\{(X_t,Y_{1,t},Y_{2,t}){\}}_{t=1}^n\text{ i.i.d. }{\bar{P}}_{X{Y}_1{Y}_2}$$

$${\varphi }_{1,n}:{\mathcal{X}}^n\to {\mathcal{M}}_1\triangleq \{0,1,\dots ,{\mathsf{W}}_{1,n}-1\}$$

$$M_1={\varphi }_{1,n}(X^n)$$

$${\varphi }_{2,n}:{\mathcal{M}}_1\times {\mathcal{Y}}_1^n\to {\mathcal{M}}_2\triangleq \{0,1,\dots ,{\mathsf{W}}_{2,n}-1\},$$

$${\psi }_{1,n}:{\mathcal{M}}_1\times {\mathcal{Y}}_1^n\to \{0,1\}.$$

$$M_2={\varphi }_{2,n}(M_1,Y_1^n)$$

$${\hat{\mathcal{H}}}_1\triangleq {\psi }_{1,n}(M_1,Y_1^n).$$

$${\psi }_{2,n}:{\mathcal{M}}_1\times {\mathcal{M}}_2\times {\mathcal{Y}}_2^n\to \{0,1\}$$

$${\hat{\mathcal{H}}}_2\triangleq {\psi }_{2,n}(M_1,M_2,Y_2^n).$$

$${\alpha }_{1,n}$$

$${\beta }_{1,n}$$

$${\alpha }_{2,n}$$

$${\beta }_{2,n}$$

$$\underset{n\to \infty }{\overline{\lim }}{\alpha }_{1,n}\le 0,\underset{n\to \infty }{\overline{\lim }}{\alpha }_{2,n}\le 0,$$

$${\theta }_1$$

$$\underset{n\to \infty }{\overline{\lim }}\frac{1}{n}\log {\mathsf{W}}_{1,n}$$

$$R_1=R_2=0,$$

$${\mathsf{W}}_{1,n}$$

$${\mathsf{W}}_{2,n}$$

$${\bar{P}}_{X{Y}_1{Y}_2}=P_{X{Y}_1}{P}_{Y_2}.$$

$$P_{X{Y}_1{Y}_2}$$

$${\bar{P}}_{X{Y}_1{Y}_2}$$

$$P_{X{Y}_1{Y}_2}$$

$${\bar{P}}_{X{Y}_1{Y}_2}$$

$${\theta }_1$$

$${\theta }_2$$

$${\theta }_1$$

$${\theta }_2$$

$${\mathsf{W}}_1\ge 3\text{and}{\mathsf{W}}_2\ge 2,$$

$${\mathcal{T}}_{\mu }^n(P_X)\cap {\mathcal{T}}_{\mu }^n({\bar{P}}_X)=\varnothing .$$

$$M_1=\{\begin{array}{cc}0& \text{ if }{x}^n\in {\mathcal{T}}_{\mu }^n(P_X)\\ 1& \text{ if }{x}^n\in {\mathcal{T}}_{\mu }^n({\bar{P}}_X)\\ 2& \text{ otherwise}.\end{array}$$

$${\hat{\mathcal{H}}}_1=\{\begin{array}{cc}1& \text{ if }{m}_1=1\text{and}{y}_1^n\in {\mathcal{T}}_{\mu }^n({\bar{P}}_{Y_1})\\ 0& \text{ otherwise}.\end{array}$$

$$M_2=\{\begin{array}{cc}0& \text{ if }{m}_1=0\text{and}{y}_1^n\in {\mathcal{T}}_{\mu }^n(P_{Y_1})\\ 1& \text{ otherwise}\end{array}$$

Full text

I Introduction

Problems of distributed hypothesis testing are strongly rooted in both statistics and information theory. In particular, [1, 2, 3] considered a distributed hypothesis testing problem where a a single sensor communicates with a single detector over a rate-limited but noise-free link. The goal of [1, 2] and [3] was to determine the largest Type-II error exponent under a fixed constraint on the Type-I error exponent. Ahlswede and Csiszár in [1] presented a coding and testing scheme and the corresponding Type-II error exponent for this problem, and established optimality of the exponent for the special case of testing against independence. For the general case, the scheme was subsequently improved by Han [2] and by Shimokawa, Han, Amari [3]. The latter scheme was shown to achieve the optimal exponent in the special case of testing against conditional independence by Rahman and Wagner [4].

This line of works has also been extended to networks with multiple sensors [2, 4, 5, 6, 7], multiple detectors [8], interactive terminals [9, 10, 11], multi-hop networks [12, 13, 14, 15, 16], noisy channels [17, 18] and scenarios with privacy constraints [19, 20, 21, 22]. The works most closely related to the present manuscript are [14, 8] and [23]. The former two, [14, 8], fully characterize the set of possible Type-II error exponents in the special case of testing against independence and testing against conditional independence for a scenario with a single sensor and two non-cooperative detectors. In this paper we consider a similar scenario but where one of the detectors can send a cooperation message to the other detector. The same configuration was already considered in [23] for a specific case of testing against independence (see Remark 1 in Section II) and under the assumption that only the detector receiving the cooperation message has to take a decision.

The results discussed so far concern scenarios where communication is of positive rates. An important line of work also assumes zero rate communication. The single-sensor single-detector version of this problem was addressed in [2] and [24], where Han identified the optimal exponent for the case where only a single bit is communicated and Shalaby and Papamarcou proved that this exponent is also optimal when communication comprises a sublinear number of bits. The finite length regime was investigated in [25]. Zero-rate hypothesis testing in interactive setup and multi-hop networks were addressed respectively in [26] and [13, 16, 27]. In particular, in our previous work [16], we proved a similar result for the single-sensor two-detectors setup. In the current manuscript we consider the extension to a cooperative setup.

Specifically, in this paper we consider the single-sensor two-detectors system in Fig. 1 where Detector 1, after receiving a message from the Sensor, can send a message to Detector 2. This additional message allows the detectors to collaborate in their decision and one of the the goal of the paper is to quantify the increase in the Type-II error exponents that is enable by this collaboration. We show that even a single bit of communication between the detectors (the temptative guess about the hypothesis at the transmitting detector) can provide an unbounded gain in the Type-II error exponent of the detector receiving the bit.

On a more technical level, the presence of a cooperation link between the detectors seems to make the problem of identifying the optimal Type-II exponents significantly more difficult. For example, without cooperation, the set of achievable exponents for testing against independence has been solved in [14], and it is achievable with a simple scheme that does not rely on binning. With cooperation, we managed to identify the optimal exponents only under the additional assumption that the observations at the two detectors are independent under both hypotheses and the cooperation rate is zero. In the general case, binning is necessary, which makes it hard to prove optimality of the achieved exponent. Notable exceptions are the results included in [4, 28, 29, 18, 6].

For the sake of simplicity, in the case of positive communication rates we therefore only present and analyze a simple coding scheme without binning and also without Heegard-Berger [30] coding. We prove that this simple scheme is optimal in a special case of testing against independence where it achieves an exponent at Detector 2 equal to the sum of the exponents at both detectors in a non-cooperative setup. Collaboration between detectors thus allows to accumulate the error exponents at the detectors. The testing against independence problem considered in this paper differs from the one in [23], where the first detector cannot achieve a positive error exponent.

In our two-detectors setup where each detector aims at maximizing the error exponent under one of the two hypotheses, two cases can be distinguished: both detectors aim at maximizing their exponents under the same hypothesis (we refer to this setup as coherent detection) or the two detectors aim at maximizing their exponents under different hypotheses (we refer to this setup as concurrent detection). In this paper we consider both scenarios. The exponents region can significantly differ under the two, in particular when based on its own observation the sensor can guess the hypothesis, communicate this guess to the detectors, and adapt the communication to this guess. With this strategy, the exponents region achieved by our simple scheme is a rectangle under concurrent detection, which means that each detector’s exponent is the same as in a setup where the other detector is not present. Under coherent detection or concurrent detection when the sensor cannot distinguish the two hypotheses, the exponents region achieved by our scheme shows a tradeoff between the two exponents. These results are for positive communication rates.

We also consider the case with fixed-length communication. Under coherent detection or concurrent detection when the sensor can send more than a single bit or cannot distinguish the two hypotheses, the exponents region is a rectangle. In these cases, each detector achieves the same exponent as if it were the only detector in the system. In contrast, a tradeoff arises if the sensor can distinguish the two hypotheses but can only send a single bit to the detectors. A comparison with the optimal exponents regions without cooperation [16], allows us to exactly quantify the benefits of detector collaboration in this setup with fixed communication alphabets. All results explained in this paragraph remain valid when the alphabets size are not fixed but grow sublinearly in the length of the observed sequences. They also generalize to an arbitrary number of hypotheses. Whereas for two detectors a tradeoff between the exponents arises only when the sensor sends a single bit to the detectors, in a multi-hypothesis testing scenario with $\mathsf{H}\geq 3$ such a tradeoff can arise whenever the number of communicated bits does not exceed $\log_{2}\mathsf{H}$.

I-A Paper Organization

The remainder of this paper is organized as follows. Section II describes the system model. Sections III and Section V describe our main results: Section III focuses on fixed communication alphabets and Section V when communication is of positive rates. For the purpose of comparison, Section IV presents results for the extreme case when communication rates are very high. Technical proofs are referred to appendices. The paper is concluded in Section VI.

I-B Notation

Throughout, we use the following notation. Random variables are denoted by capital letters and their realizations by lower case, e.g., $X$ and $x$. A random or deterministic indexed $n$-tuple $X_{1},\ldots,X_{n}$ or $x_{1},\ldots,x_{n}$ is abbreviated as $X^{n}$ or as $x^{n}$. The set of all possible types of $n$-length sequences over $\mathcal{X}$ is denoted $\mathcal{P}^{n}(\mathcal{X})$. For $\mu>0$, the set of sequences $x^{n}$ that are $\mu$-typical with respect to the pmf $P_{X}$ is denoted $\mathcal{T}^{n}_{\mu}(P_{X})$, the type of a sequence $x^{n}$ is denoted by $\text{tp}(x^{n})$. The set of all $n$-length sequences with a given type $P_{X}$, or type class, is denoted $\mathcal{T}^{n}_{0}(P_{X})$. For random variables $X$ and $\bar{X}$ over the same alphabet $\mathcal{X}$ with pmfs $P_{X}$ and $\bar{P}_{X}$ satisfying $P_{X}\ll\bar{P}_{X}$ (i.e., for every $x\in\mathcal{X}$, if $P_{X}>0$ then also $\bar{P}_{X}>0$), both $D(P_{X}\|\bar{P}_{X})$ and $D(X\|\bar{X})$ denote the Kullback-Leiber divergence between $X$ and $\bar{X}$. Finally, $H(\cdot)$ and $I(\cdot;\cdot)$ denote entropy and mutual information.

II Formal Problem Statement

Consider a three-terminal problem with a Sensor observing the sequence $X^{n}$, a Detector 1 observing $Y_{1}^{n}$, and a Detector 2 observing $Y_{2}^{n}$. The joint probability mass function (pmf) of the tuple $(X^{n},Y^{n}_{1},Y^{n}_{2})$ depends on one of two hypotheses. Under hypothesis

[TABLE]

and under hypothesis

[TABLE]

The Sensor applies an encoding function

[TABLE]

to its observed source sequence $X^{n}$ and sends the resulting index

[TABLE]

to both decoders. Detector 1 then applies two functions to the pair $(M_{1},Y_{1}^{n})$ an encoding function

[TABLE]

and a decision function

[TABLE]

It sends the index

[TABLE]

to Detector 2, and decides on the hypothesis

[TABLE]

Detector 2 applies a decision function

[TABLE]

to the triple $(M_{1},M_{2},Y_{2}^{n})$ to produce the decision

[TABLE]

Both Detectors are required to have vanishing probabilities of error under both hypotheses. Moreover, for Detector 2, we require that the probability of error under $\mathcal{H}=1$ decays exponentially fast with the largest possible exponent. For Detector 1, we consider two scenarios: coherent detection and concurrent detection. Under coherent detection, Detector 1 wishes to maximize the exponential decay of the probability of error under $H=1$. Under concurrent detection, Detector 1 wishes to maximize the exponential decay of the probability of error under $H=0$. In an unifying manner, we define, for $h_{1}\in\{0,1\}$ and $\bar{h}_{1}=(h_{1}+1)\textnormal{ mod }2$, the following error probabilities:

[TABLE]

Definition 1** (Achievability under Rate-Constraints)**

Given $\bar{h}_{1}\in\{0,1\}$ and rates $R_{1},R_{2}\geq 0$, an error-exponents pair $(\theta_{1},\theta_{2})$ is said achievable if for all blocklengths $n$ there exist functions $\phi_{1,n}$, $\phi_{2,n}$, $\psi_{1,n}$ and $\psi_{2,n}$ as in (3), (5), (6), and (9) so that the following limits hold:

[TABLE]

[TABLE]

and

[TABLE]

Definition 2** (Error-Exponents Region under Rate-Constraints)**

*For $\bar{h}_{1}\in\{0,1\}$ and rates $R_{1},R_{2}\geq 0$ the closure of the set of all achievable exponent pairs $(\theta_{1},\theta_{2})$ is called the error-exponents region $\mathcal{E}(R_{1},R_{2})$. *

When both rates are zero,

[TABLE]

we are also interested in determining the exponents region with fixed communication alphabets of sizes:

[TABLE]

Definition 3** (Achievability with Fixed Communication Alphabets)**

*For $\bar{h}_{1}\in\{0,1\}$ and communication alphabet sizes $\mathsf{W}_{1},W_{2}\geq 0$, an error-exponents pair $(\theta_{1},\theta_{2})$ is said achievable if for all blocklengths $n$ there exist functions $\phi_{1,n}$, $\phi_{2,n}$, $\psi_{1,n}$ and $\psi_{2,n}$ as in (3), (5), (6), and (9) so that (19) and the limits in (15) and (16) hold. *

Definition 4** (Error-Exponents Region for Fixed Communication Alphabets)**

*For fixed $\bar{h}_{1}\in\{0,1\}$ and communication alphabet sizes $\mathsf{W}_{1},\mathsf{W}_{2}\geq 0$, the closure of the set of all achievable exponent pairs $(\theta_{1},\theta_{2})$ is called the error-exponents region $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2})$. *

Remark 1

The model of Fig. 1 was also considered in [23] in a special case of testing against independence where

[TABLE]

Moreover, in [23] only Detector 2 guesses the binary hypothesis but not Detector 1.

II-A Some Degenerate Cases

In some special cases, the described setup degenerates and the error-exponents region is the same as in a setup without cooperation or in a setup with a single centralized detector.

We first consider a setup where cooperation is not beneficial.

Proposition 1

Assume the Markov chain $X-\!\!\!\minuso\!\!\!-Y_{2}-\!\!\!\minuso\!\!\!-Y_{1}$ under both hypotheses with identical law $P_{Y_{1}|Y_{2}}=\bar{P}_{Y_{1}|Y_{2}}$:

[TABLE]

In this case, irrespective of the cooperation rate $R_{2}\geq 0$ and of the value of $\bar{h}_{1}\in\{0,1\}$, the exponent region $\mathcal{E}(R_{1},R_{2})$ and $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2})$ coincides with the exponent region of the scenario without cooperation (see Fig. 2).

Proof:

The exponents regions $\mathcal{E}(R_{1},R_{2})$ and $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2})$ of the original setup cannot be larger than the exponent regions of an enhanced setup (with cooperation) where Detector 2 not only observes $Y_{2}^{n}$ but also $Y_{1}^{n}$. But in this new setup, the cooperation link is useless because Detector 2 can generate the cooperation message locally. Moreover, without cooperation, the observation $Y_{1}^{n}$ is not beneficial because the conditional laws $P_{Y_{1}|XY_{2}}$ and $\bar{P}_{Y_{1}|XY_{2}}$ coincide and only depend on $Y_{2}$, and so Detector 2 can generate a statistically equivalent observation to $Y_{1}^{n}$ itself based only on $Y_{2}^{n}$. By these arguments, the exponents regions $\mathcal{E}(R_{1},R_{2})$ and $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2})$ of the original setup are not larger than the one of the setup without cooperation. But $\mathcal{E}(R_{1},R_{2})$ and $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2})$ can also not be smaller than the exponents regions of the same setup but without cooperation, because the latter setup can be mimicked in the former. This concludes the proof. ∎

We now consider a setup that is equivalent to a setup with a single centralized detector.

Proposition 2

Assume the Markov chain $X-\!\!\!\minuso\!\!\!-Y_{1}-\!\!\!\minuso\!\!\!-Y_{2}$ holds under both hypotheses with identical law $P_{Y_{2}|Y_{1}}=\bar{P}_{Y_{2}|Y_{1}}$. I.e.,

[TABLE]

In this case, irrespective of the cooperation rate $R_{2}\geq 0$ and of the value of $h_{1}\in\{0,1\}$, the exponent regions $\mathcal{E}(R_{1},R_{2})$ and $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2})$ coincide with the exponents regions of the scenario in Fig. 2 without cooperation and where both Detectors observe $Y_{1}^{n}$ but not $Y_{2}^{n}$. As a consequence, they also coincide with the exponents regions of the scenario in Fig. 3 with a single Detector observing $Y_{1}^{n}$ that takes both decisions $\hat{\mathcal{H}}_{1}$ and $\hat{\mathcal{H}}_{2}$.

Proof:

The exponents regions $\mathcal{E}(R_{1},R_{2})$ and $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2})$ of the original setup cannot be larger than the exponents regions of an enhanced setup (with cooperation) where Detector 2 not only observes $Y_{2}^{n}$ but also $Y_{1}^{n}$. Since Detector 2 can generate an observation that is statistically equivalent to $Y_{2}^{n}$ given $Y_{1}^{n}$, the exponents regions are no larger than in the setup where both detectors observe $Y_{1}^{n}$ but not $Y_{2}^{n}$. Furthermore, since allowing the two detectors to fully cooperate in their decision can only increase the exponents regions, the regions $\mathcal{E}(R_{1},R_{2})$ and $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2})$ must be included in the exponents regions of the setup in Fig. 3 where a single detector takes both decisions.

On the other hand, $\mathcal{E}(R_{1},R_{2})$ and $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2})$ can also not be smaller than the exponents regions of the setup in Fig. 3. In fact, in the original setup, Detector $1$ can mimick the single central detector and forward the decision $\hat{H}_{2}$ to Detector $2$, which follows this decision. This strategy requires only a single cooperation bit and can thus be implemented irrespective to the available cooperation rate $R_{2}\geq 0$. This conclude the proof. ∎

III Results for Fixed Communication Alphabets

We start by presenting our results for the fixed-alphabets case, so we assume (19) and are interested in the error-exponents region $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2})$. For simplicity, we assume that $P_{XY_{1}}(x,y_{1})>0$ and $\bar{P}_{XY_{1}Y_{2}}(x,y_{1},y_{2})>0$ for all $(x,y_{1},y_{2})\in\mathcal{X}_{1}\times\mathcal{Y}_{2}\times\mathcal{Y}_{2}$.

Our main finding in this section is the exact characterization of the optimal exponents region $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2})$ for all possible parameters.

III-A Coherent Detection and Concurrent Detection with $P_{X}=\bar{P}_{X}$

In this sense, the following Propositions 3–5 are rather straightforward and we omit most of their proofs. Proposition 4 is the main result of this section.

Proposition 3** (Coherent Detection)**

For coherent detection, $\bar{h}_{1}=1$, and for all values $\mathsf{W}_{1}\geq 2$ and $\mathsf{W}_{2}\geq 2$, the exponents region $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2})$ is the set of all nonnegative rate pairs $(\theta_{1},\theta_{2})$ satisfying

[TABLE]

Proof:

The achievability follows by a similar scheme as in [2] where the sensor and Detector $1$ check whether their observed sequences are typical or not according to the P-distribution and send the message 1 if so, and 0 if not. The detectors declare $\mathcal{H}=0$ if all received messages are ones and their observed sequence is typical. Otherwise, they declare $\mathcal{H}=1$. The converse to (23) follows directly from [24]. The converse to (24) is similar and proved for completeness in Appendix A. ∎

Proposition 4** (Concurrent Detection with $P_{X}=\bar{P}_{X}$)**

Under concurrent detection, i.e., $\bar{h}_{1}=0$, and when $P_{X}=\bar{P}_{X}$, then for all values $\mathsf{W}_{1}\geq 2$ and $\mathsf{W}_{2}\geq 2$, the exponents region $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2})$ is the set of all nonnegative rate pairs $(\theta_{1},\theta_{2})$ satisfying

[TABLE]

Proof:

The achievability follows by a slight generalization of the previous scheme. The idea is that Detector 1 decides on $\mathcal{H}=1$ only if $X^{n}$ and $Y_{1}^{n}$ are typical according to the $\bar{P}$ distributions. Detector 2 decides on $\mathcal{H}=0$ only if $X^{n}$, $Y_{1}^{n}$, and $Y_{2}^{n}$ are typical according to the $P$ distributions. Since $P_{X}=\bar{P}_{X}$ this information can be conveyed to both detectors whenever $\mathsf{W}_{1},\mathsf{W}_{2}\geq 2$. The converse is similar to the one for Proposition 3 and omitted. ∎

III-B Concurrent Detection with $P_{X}\neq\bar{P}_{X}$

We now consider concurrent detection, $\bar{h}_{1}=0$, and $P_{X}\neq\bar{P}_{X}$. Here the optimal exponents region depends on whether the alphabet size $\mathsf{W}_{1}$ equals $2$ or is larger. We first assume

[TABLE]

and present a coding scheme for this scenario. Pick a small positive number $\mu>0$ such that the typical sets $\mathcal{T}_{\mu}^{n}(P_{X})$ and $\mathcal{T}_{\mu}^{n}(\bar{P}_{X})$ do not intersect:

[TABLE]

Sensor: Given that it observes $X^{n}=x^{n}$, it sends

[TABLE]

Detector 1: Given that it observes $Y_{1}^{n}=y_{1}^{n}$ and $M_{1}=m_{1}$, it decides

[TABLE]

It sends

[TABLE]

to Detector 2.

Detector 2: Given that it observes $Y_{2}^{n}=y_{2}^{n}$ and messages $M_{1}=m_{1}$ and $M_{2}=m_{2}$, it decides

[TABLE]

Proposition 5** (Concurrent Dection, $P_{X}\neq\bar{P}_{X}$ and $\mathsf{W}_{1}\geq 3$)**

Under concurrent detection and for all values $\mathsf{W}_{1}\geq 3$ and $\mathsf{W}_{2}\geq 2$, the exponents region $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2})$ is the set of all nonnegative rate pairs $(\theta_{1},\theta_{2})$ satisfying

[TABLE]

Proof:

The achievability follows by the above coding scheme; and the converse is similar to that of Proposition 3. ∎

The exponents region $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2})$ in these first three Propositions 3–5 is rectangular, and each of the detectors can simultaneouly achieve the optimal exponent as if it were the only detector in the system. As we see in the following, this is not always the case.

In the rest of this section, we assume

[TABLE]

and present the optimal exponents region for this case. It is achieved by the following coding scheme.

Pick a real number $r$, a small positive number $\mu>0$ satisfying (28), and the function $b\colon\{0,1\}\to\{0,1\}$ either as

[TABLE]

or as

[TABLE]

We then assign each type $\pi\in\mathcal{P}(\mathcal{X}^{n})$ that satisfies

[TABLE]

to one of two sets $\Gamma_{0}$ or $\Gamma_{1}$. If $b(0)=b(1)=0$, then we assign all these types to the set $\Gamma_{1}$. Otherwise, we assign them between the two sets according to the following rule:

[TABLE]

and $\pi\in\Gamma_{b(0)}$ otherwise. Given that it observes $X^{n}=x^{n}$, it sends

[TABLE]

Detector 1: Given that it observes $Y_{1}^{n}=y_{1}^{n}$ and $M_{1}=m_{1}$, it decides

[TABLE]

It sends

[TABLE]

to Detector 2.

Detector 2: Given that it observes $Y_{2}^{n}=y_{2}^{n}$ and messages $M_{1}=m_{1}$ and $M_{2}=m_{2}$, it decides

[TABLE]

The described scheme achieves the following optimal error-exponents region.

Theorem 6** (Concurrent Detection, $P_{X}\neq\bar{P}_{X}$, and $\mathsf{W}_{1}=2$)**

Under concurrent detection and for all values $\mathsf{W}_{1}=2$ and $\mathsf{W}_{2}\geq 2$, the exponents region $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2})$ is the set of all nonnegative rate pairs $(\theta_{1},\theta_{2})$ that satisfy

[TABLE]

for some real $r$ and one of the mappings in (36) and (37), and the corresponding sets $\Gamma_{0}$ and $\Gamma_{1}$.

Proof:

The achievability follows by the described scheme, by Sanov’s theorem, and by noting that $\hat{\mathcal{H}}_{1}=1$ iff,

[TABLE]

whereas $\hat{\mathcal{H}}_{2}=0$, if, and only if,

[TABLE]

The converse is proved in Appendix B. ∎

Remark 2** (Sending a sublinear number of bits)**

A close inspection of the converse proofs for Propositions 3–5 shows that they remain valid when the alphabet sizes are not fixed but grow sublinearly in the blocklength $n$, i.e., when

[TABLE]

Remark 3** (Extension to many hypotheses)**

Most of the results in this section can easily be extended to a scenario with more than two hypotheses. For $\mathsf{H}=2$ the exponents region showed a tradeoff in the exponents under concurrent detection only when $\mathsf{W}_{1}=\mathsf{W}_{2}=2$. In contrast, for $\mathsf{H}\geq 3$, a tradeoff arises for a variety of pairs $\mathsf{W}_{1},\mathsf{W}_{2}$. In general, the minimum required values for $\mathsf{W}_{1}$ and $\mathsf{W}_{2}$ leading to a rectangular exponents region coincides respectively with the number of hypotheses which have distinct $X$-marginals and the number of hypothesis which have distinct $Y_{1}$-marginals.

III-C Benefits of Cooperation

To discuss the benefits of cooperation, we quickly state the optimal exponents region without cooperation, i.e., for111Equivalently, the no cooperation setup could be parametrized as $\mathsf{W}_{2}=1$.

[TABLE]

They were determined in [16]. Under coherent detection or under concurrent detection with $P_{X}=\bar{P}_{X}$ or $\mathsf{W}_{1}\geq 3$, the exponents region $\mathcal{E}_{0}(\mathsf{W}_{1},\mathsf{W}_{2}=0)$ are similar to Propositions 3–5 but with a modified constraint on $\theta_{2}$. More precisely, Propositions 3–5 remain valid for $\mathsf{W}_{2}=0$ if the constraints on $\theta_{2}$, (24), (26), (34) are replaced by

[TABLE]

So, in these scenarios, the exponents region is a rectangle both in the case with and without cooperation, and with cooperation the $\theta_{2}$-side of the rectangle is increased by the quantity

[TABLE]

Under concurrent detection when $P_{X}\neq\bar{P}_{X}$ and $\mathsf{W}_{1}=2$, the exponents region is not a rectangle, but there is a tradeoff between the two exponents. In this case, it seems difficult to quantify the cooperation benefit in general.

We now present an example for concurrent detection with $P_{X}\neq\bar{P}_{X}$.

Example 1

Consider a setup where $X,Y_{1},Y_{2}$ are binary with pmfs

[TABLE]

Fig. 4 illustrates the exponents region for coherent and concurrent detection of Propositions 3 and 5 and of Theorem 6. Specifically, the figure on the left shows the exponents region with or without cooperation under coherent detection, and the figure on the right shows these exponents regions under concurrent detection. Under concurrent detection the exponents regions for $\mathsf{W}_{1}=3$ and $\mathsf{W}_{1}=2$ are shown.

IV High-Rates Regime

We now consider the other (trivial) extreme case where both links are of high rates so that under hypothesis $\mathcal{H}=h_{1}$, Detector 1 can learn the sequence $X^{n}$ with high probability and under $\mathcal{H}=0$, Detector 2 can learn both sequences $X^{n}$ and $Y_{1}^{n}$ with high probability. We will see that in this case both Detecor 1 attains the exponent of a centralized setup where it observes $(X^{n},Y_{1}^{n})$ and Detector 2 attains the exponent of a centralized setup where it observes $(X^{n},Y_{1}^{n},Y_{2}^{n})$.

We first consider coherent detection where $\bar{h}_{1}=1$. Pick a small $\epsilon$. The Sensor describes the sequence $X^{n}$ to both detectors if $X^{n}\in\mathcal{T}_{\mu}^{(n)}(P_{X})$, and otherwise it sends [math]. Detector 1 describes the sequence $Y_{1}^{n}$ to Detector 2 if $(X^{n},Y_{1}^{n})\in\mathcal{T}_{\mu}^{(n)}(P_{XY_{1}})$, and otherwise it sends 0. The described coding scheme requires rates

[TABLE]

Detector 1 decides on $\hat{\mathcal{H}}_{1}=1$, if the Sensor sent [math] or itself it sent [math]. Otherwise it decides on $\hat{\mathcal{H}}_{1}=1$. Detector 2 decides on $\hat{\mathcal{H}}_{2}=1$, if the Sensor or Detector 1 sent [math]. Otherwise it decides on $\hat{\mathcal{H}}_{1}=1$ if and only if its own observation $Y_{2}^{n}$ and the received sequences $X^{n}$ and $Y_{1}^{n}$ are jointly typical, $(X^{n},Y_{1}^{n},Y_{2}^{n})\in\mathcal{T}_{\mu}^{(n)}(P_{XY_{1}Y_{2}})$.

The described scheme achieves the set of all nonnegative pairs $(\theta_{1},\theta_{2})$ satisfying

[TABLE]

This set coincides with the optimal exponents region $\mathcal{E}(R_{1},R_{2})$, because it also coincides with the exponent region of a centralized setup where Detector 1 observes both $X^{n}$ and $Y_{1}^{n}$ and Detector 2 observes all $X^{n}$, $Y_{1}^{n}$, and $Y_{2}^{n}$.

Consider now concurrent detection where $\bar{h}_{1}=0$. In this case, the Sensor describes the sequence $X^{n}$ to both Detectors if $X^{n}\in\mathcal{T}_{\mu}^{(n)}(P_{X})$ or if $X^{n}\in\mathcal{T}_{\mu}^{(n)}(\bar{P}_{X})$. Otherwise it sends [math]. Detector 1 describes the sequence $Y_{1}^{n}$ to Detector 2 if $(X^{n},Y_{1}^{n})\in\mathcal{T}_{\mu}^{(n)}(P_{XY_{1}})$, and otherwise it sends 0. Detector 2 decides as above and Detector 1 decides on $\hat{\mathcal{H}}_{1}=0$ if and only if its own observation $Y_{1}^{n}$ and the described sequence $X^{n}$ are jointly typical, i.e., $(X^{n}Y_{1}^{n})\in\mathcal{T}_{\mu}^{n}(\bar{P}_{XY_{1}})$. The coding scheme requires rates

[TABLE]

and achieves the set of all nonnegative pairs $(\theta_{1},\theta_{2})$ satisfying

[TABLE]

Again, this set coincides with the optimal exponents region $\mathcal{E}(R_{1},R_{2})$ because it also coincides with the optimal exponents region when Detector 1 observes the pair $X^{n},Y_{1}^{n}$ and Detector 2 observes $X^{n}$, $Y_{1}^{n}$, and $Y_{2}^{n}$.

Both results remain valid without cooperation if the term $D(P_{XY_{1}Y_{2}}\|\bar{P}_{XY_{1}Y_{2}})$ limiting the second exponent $\theta_{2}$ is replaced by $D(P_{XY_{2}}\|\bar{P}_{XY_{2}})$. The benefit of cooperation is thus equal to $\mathbb{E}_{P_{XY_{2}}}[D(P_{Y_{1}|XY_{2}}\|\bar{P}_{Y_{1}|XY_{2}})]$ in both cases.

V Results for Positive Rates $R_{1}>0$

In this section, we assume that

[TABLE]

The cooperation rate $R_{2}$ can be [math] or larger.

V-A Testing Against Independence under Coherent Detection

We start with the special case of “testing-against-independence” scenario under coherent detection, $\bar{h}_{1}=1$, where

[TABLE]

Notice that this setup differs from the testing-against independence scenario in [23] (see also Remark 1).

We assume a cooperation rate $R_{2}=0$, which means that Detector can send a message $M_{2}$ to Detector 2 that is described by a sublinear number of bits.

The simple scheme in the next subsection V-B achieves the following exponents region, which can be proved to be optimal.

Theorem 7** (Testing Against Independence)**

Assume $\bar{h}_{1}=1$ and (68). Then, $\mathcal{E}(R_{1},0)$ is the set of all nonnegative exponent pairs $(\theta_{1},\theta_{2})$ for which

[TABLE]

for some $U$ satisfying the Markov chain $U-\!\!\!\minuso\!\!\!-X-\!\!\!\minuso\!\!\!-(Y_{1},Y_{2})$ and the rate constraint $R_{1}\geq I(U;X)$.

Proof:

The achievability follows by specializing and evaluating Theorem 9 for this setup. The converse is proved in Appendix C. ∎

Lemma 8** (Cardinality bound)**

the right hand sides of (70) in Theorem  7 remain valide if we impose the cardinality bound $\|\mathcal{U}\|=\|\mathcal{X}\|+1$.

Proof:

see [2, Theorem 3]. ∎

Notice that for $R_{2}=0$ the scheme in the following subsection sends only a single bit and that without cooperation, the term $I\left(U;Y_{2}\right)$ needs to be removed on the right-hand side of (70b). This mutual information term thus represents the benefit of a single cooperation bit from Detector 1 to Detector 2.

We illustrate the benefit of cooperation at hand of the following example.

Example 2

Consider a setup with coherent detection, $\bar{h}_{1}=1$, where $X,Y_{1},Y_{2}$ are ternary and under $\mathcal{H}=0$:

[TABLE]

whereas under $\mathcal{H}=1$ they are independent with same marginals as under $\mathcal{H}=0$.

Fig. 5 illustrates an achievable error-exponents region obtained with theorem 9 when the communication rate are $R_{1}=0.4\textnormal{ bits}$ and $R_{2}=1.0\textnormal{ bits}$. It also shows the error-exponents region $\mathcal{E}(0.4,0)$ presented in Theorem 7 and the error-exponents region without cooperation when $R_{1}=0.4\textnormal{ bits}$ derived in [14, Theorem 1]. Notice that the scheme achieving $\mathcal{E}(0.4,0)$ requires that only a single cooperation bit is sent from Detector 1 to Detector 2.

V-B A Simple Scheme with Cooperation

In this subsection, we present coding schemes for both coherent and concurrent detection. Our schemes do not use binning or Heegard-Berger coding because of the complexity of the expressions describing the error exponents achieved with such schemes. In fact, such an approach would lead to expressions with 18 competing exponents, see the supplementary material for the case without cooperation. Notice that this difficulty seems inherent to all multi-user hypothesis testing scenarios, see e.g. [15]. We first present a scheme for coherent detection,

[TABLE]

and then explain how to change the scheme for concurrent detection.

V-B1 Coherent Detection

Preliminaries: Fix a small $\delta>0$ and a pair of auxiliary random variables $(U,V)$ so that the following Markov chains

[TABLE]

and satisfying the rate constraints

[TABLE]

Codebook Generation: we randomly generate the codebook

[TABLE]

by drawing each entry of each codeword $u^{n}(m_{1})$ i.i.d. according to $P_{U}$.

Furthermore, we superpose a codebook $\mathcal{C}_{V}$ on codebook $\mathcal{C}_{U}$. So, for each index $m_{1}\in\{1,\ldots,\lfloor 2^{nR_{1}}\rfloor\}$, we randomly construct the codebook

[TABLE]

by drawing the $j$-th entry of each codeword $v^{n}(m_{2}|m_{1})$ according to the conditional pmf $P_{V|U}(\cdot|u_{j}(m_{1}))$, where $u_{j}(m_{1})$ denotes the $j$-th component of codeword $u^{n}(m_{1})$.

Reveal all codebooks to all terminals.

Sensor: Assume it observes the source sequence $X^{n}=x^{n}$. Then, it first looks for a message $m_{1}\in\{1,\ldots,\lfloor 2^{nR_{1}}\rfloor\}$ such that

[TABLE]

If one or multiple such indices $m$ are found, the Sensor selects $m^{*}$ uniformly at random over these indices and sends

[TABLE]

Otherwise, it sends

[TABLE]

Detector 1: If

[TABLE]

Detector 1 decides on the alternative hypothesis

[TABLE]

If

[TABLE]

and given that $Y_{1}^{n}=y_{1}^{n}$, Detector 1 checks whether

[TABLE]

If the test is successful, it decides on the null hypothesis

[TABLE]

Otherwise it decides on the alternative hypothesis as in (85).

We now describe the communication to Detector 2. If

[TABLE]

Detector 1 sends

[TABLE]

Otherwise, it looks for an index $m_{2}\in\{1,\ldots,\lfloor 2^{nR_{2}}\rfloor\}$ such that

[TABLE]

If one or more such indices can be found, Detector 1 selects an index $m_{2}^{*}$ among them uniformly at random and sends

[TABLE]

Otherwise it sends $M_{2}=0$.

Detector 2: If

[TABLE]

Detector 2 decides on the alternative hypothesis

[TABLE]

If

[TABLE]

and given $Y_{2}^{n}=y_{2}^{n}$, Detector 2 checks whether

[TABLE]

If this check is successful, Decoder 2 decides on the null hypothesis

[TABLE]

Otherwise, it decides on the alternative hypothesis

[TABLE]

V-B2 Changes for concurrent detection when $P_{X}=\bar{P}_{X}$

We now consider the scenario of concurrent detection, so

[TABLE]

We apply the same scheme as above, except for the decision at Detector 1, which is described next.

Detector 1: If

[TABLE]

Detector 1 now decides

[TABLE]

If

[TABLE]

and given that $Y_{1}^{n}=y_{1}^{n}$, Detector 1 checks whether

[TABLE]

If the test is successful, it decides

[TABLE]

Otherwise it decides

[TABLE]

Communication to Detector 2 is as described in the previous subsection.

V-B3 Changes for concurrent detection when $P_{X}\neq\bar{P}_{X}$

If $\bar{h}_{1}=0$ and $P_{X}\neq\bar{P}_{X}$, the scheme should be changed as described in the previous paragraph. The following additional changes allow to obtain an improved scheme.

In this case, we choose $\mu>0$ so that the intersection

[TABLE]

is empty and we choose another auxiliary random variable $\bar{U}_{1}$ satisfying

[TABLE]

A third codebook

[TABLE]

is drawn by picking the entries i.i.d. according to $\bar{P}_{U_{1}}$.

Encoding has to be changed as follows. If the test in (91) fails, then the Sensor looks for an index $m_{1}\in\{1,\ldots,\lfloor 2^{nR_{1}}\rfloor\}$ such that

[TABLE]

If one or multiple such indices $m_{1}$ are found, the Sensor selects $m_{1}^{*}$ uniformly at random over these indices and sends

[TABLE]

Notice that by the condition (106), only one of the two tests (91) and (110) can be successful for any observed sequence $x^{n}$. It therefore does not matter which one is performed first.

Thus, now the Sensor sends three different types of messages:

[TABLE]

The message $M_{1}=(1,m_{1})$ indicates that the Sensor is tempted to guess $\mathcal{H}=0$. After receiving such a message, Detector 1 therefore produces $\hat{\mathcal{H}}_{1}=0$. The same holds if $M_{1}=0$. In contrast, if $M_{1}=(2,m_{1})$, Detector 1 checks whether

[TABLE]

If successful it declares $\hat{\mathcal{H}}_{1}=1$, and otherwise $\hat{\mathcal{H}}_{1}=0$.

Communication from, Detector 1 to Detector 2 is as described before.

Similarly, the message $M_{1}=(2,m_{1})$ now indicates that the Sensor is tempted to guess $\mathcal{H}=1$. When receiving this message, Detector 2 therefore decides immediately $\hat{\mathcal{H}}_{2}=1$. Otherwise it acts as described in the original scheme.

V-C Achievable regions

We now present the regions achieved by the coding scheme described in the previous subsection. Notice that the new achievable regions recover the extreme cases in the previous section IV, when the rates are set accordingly.

We first consider coherent detections $\bar{h}_{1}=1$.

For given rates $R_{1}\geq 0$ and $R_{2}\geq 0$, define the following set of auxiliary random variables:

[TABLE]

Further, define for each $(U,V)\in\mathcal{S}\left(R_{1},R_{2}\right)$, the sets

[TABLE]

and

[TABLE]

and the random variables $(\bar{U},\bar{V})$ so as to satisfy

[TABLE]

and the Markov chains

[TABLE]

Theorem 9** (Coherent Detection)**

If

[TABLE]

the exponents region $\mathcal{E}(R_{1},R_{2})$ contains all nonnegative pairs $(\theta_{1},\theta_{2})$ that satisfy:

[TABLE]

for some $(U,V)\in\mathcal{S}\left(R_{1},R_{2}\right)$

Proof:

The exponent region is achieved by the scheme described in Subsection V-B. The proof is given in Appendix D. ∎

For our second result, we also define for each auxiliary random variable $U$ the set

[TABLE]

Theorem 10** (Concurrent Detection with $P_{X}=\bar{P}_{X}$)**

If

[TABLE]

then the exponents region $\mathcal{E}(R_{1},R_{2})$ contains all nonnegative pairs $(\theta_{1},\theta_{2})$ that for some $(U,V)\in\mathcal{S}\left(R_{1},R_{2}\right)$ satisfy:

[TABLE]

Proof:

Similar to the proof of Theorem 9 and omitted. ∎

For a given rate $R_{1}\geq 0$, define the following set of auxiliary random variables:

[TABLE]

and the random variable $U_{1}$ so that $P_{U_{1}|X}=\bar{P}_{U_{1}|X}$ and the Markov chain $U_{1}-\!\!\!\minuso\!\!\!-X-\!\!\!\minuso\!\!\!-Y_{1}$ holds.

Theorem 11** (Concurrent Detection and $P_{X}\neq\bar{P}_{X}$)**

If

[TABLE]

then the exponents region $\mathcal{E}(R_{1},R_{2})$ contains all nonnegative pairs $(\theta_{1},\theta_{2})$ that for some $(U,V)\in\mathcal{S}\left(R_{1},R_{2}\right)$ and $\bar{U}_{1}\in\mathcal{S}_{\textnormal{nc}}(R_{1})$ satisfy:

[TABLE]

Proof:

The proof is given in Appendix E. It is based on the scheme of the previous Subsection V-B. ∎

Remark 4

As seen in Theorem 7, in some special case exponents accumulates.

Remark 5

The exponents region in Theorem 11 is rectangular because $\theta_{1}$ depends only on the auxiliary $\bar{U}_{1}$ and $\theta_{2}$ only on the pair of auxiliaries $(U,V)$. This implies that both exponents can be maximized at the same time without any tradeoff between the two exponents.

This is different in the first two Theorem 9 and 10 where both exponents depend on the same auxiliary, and therefore the regions exhibit a tension when maximizing the two exponents.

VI Summary and Conclusion

In this paper we investigated the role of cooperation under both coherent and concurrent detection in a two-detector hypothesis testing system. We characterized fully the exponents region for fixed communication alphabets and in a testing-against independence scenario with positive communication rates. For the general positive-rate scenario, we proposed a simple scheme in which Detector 1 uses the cooperation link to inform Detector 2 about its guess and a compressed version of its observations. Our scheme behaves differently for coherent and for concurrent detection. In fact, for coherent detection, if possible the sensor guesses the hypothesis, conveys this guess to the detectors, and then focuses on helping the detector that wishes to maximize the exponent under the hypothesis not corresponding to its guess.

Our results allowed us to exactly quantify the gains of cooperation for communication with finite alphabets and for the testing against independence scenario solved in this paper. Cooperation gains are not bounded even with only a single bit of cooperation.

Depending on the specific scenario (coherent or concurrent detection, positive or zero rate, $P_{X}=\bar{P}_{X}$ or $P_{X}\neq\bar{P}_{X}$), the exponents achieved at the two detectors may or may not show a tradeoff. The absence of such a tradeoff implies that each detector can achieve the same exponent as if it were the only detector in the system.

Acknowledgement

M. Wigger acknowledges funding support from the European Research Council under grant agreement 715111.

Appendix A Proof of the Converse Bound (24) in Proposition 3

Fix an achievable Type-II error exponent $\theta_{2}$. Then choose a small number $\epsilon>0$, a sufficiently large blocklength $n$, and encoding and decision functions $\phi_{1,n},\phi_{2,n},\psi_{2,n}$ satisfying

[TABLE]

and

[TABLE]

For the chosen encoding and decision functions, define for each pair $(m_{1},m_{2})\in\{0,\ldots,\mathsf{W}_{1}-1\}\times\{0,\ldots,\mathsf{W}_{2}-1\}$ the subsets

[TABLE]

Notice that the sets $\mathcal{C}_{0},\ldots,\mathcal{C}_{W_{1}-1}$ partition $\mathcal{X}^{n}$ and for each $m_{1}\in\{0,\ldots,\mathsf{W}_{1}-1\}$ the sets $\mathcal{G}_{m_{1},0},\ldots,\mathcal{G}_{m_{1},\mathsf{W}_{2}-1}$ partition $\mathcal{Y}_{1}^{n}$. Moreover, the acceptance region $\mathcal{A}^{2}_{n}$ at Detector $2$, defined through the relation

[TABLE]

can be expressed as

[TABLE]

By the constraint on the type-I error probability on Detector $2$, (136),

[TABLE]

Now, by the union bound there exists an index pair $(m_{1}^{*},m_{2}^{*})\in\{0,\ldots,\mathsf{W}_{1}-1\}\times\{0,\ldots,\mathsf{W}_{2}-1\}$ such that:

[TABLE]

Combining (144) with an extension of [24, Theorem 3] to three pmfs (recall that we assumed $P_{XY_{1}Y_{2}}(x,y_{1},y_{2})>0$ and thus $P_{XY_{1}Y_{2}}\ll\bar{P}_{XY_{1}Y_{2}}$), for sufficiently large $n$, we obtain:

[TABLE]

Taking $n\to\infty$ and $\mu\to 0$, by the continuity of KL-divergence, we can conclude that for any achievable exponent $\theta_{2}$:

[TABLE]

This conclude the proof.

Appendix B Converse To Theorem 6

Fix a real number $r$ and an exponent pair $(\theta_{1},\theta_{2})\in\mathcal{E}_{0}(2,2)$ satisfying

[TABLE]

Then fix a small number $\epsilon>0$, a sufficiently large blocklength $n$, and encoding and decision functions $\phi_{1,n},\phi_{2,n},\psi_{1,n},\psi_{2,n}$ satisfying

[TABLE]

and

[TABLE]

For the chosen encoding and decision functions, define for each $m_{1}\in\{0,1,\ldots,\mathsf{W}_{1}-1\}$ and $m_{2}\in\{0,1,\ldots,\mathsf{W}_{2}-1\}$, the subsets

[TABLE]

Notice that the sets $\mathcal{C}_{0},\ldots,\mathcal{C}_{W_{1}-1}$ partition $\mathcal{X}^{n}$ and for each $m_{1}\in\{0,\ldots,\mathsf{W}_{1}-1\}$ the sets $\mathcal{G}_{m_{1},0},\ldots,\mathcal{G}_{m_{1},\mathsf{W}_{2}-1}$ partition $\mathcal{Y}_{1}^{n}$. Moreover, the acceptance regions $\mathcal{A}^{1}_{n}$ and $\mathcal{A}^{2}_{n}$ at Detectors $1$ and $2$, defined through the relations

[TABLE]

can be expressed as

[TABLE]

Define now for each $m_{1}\in\{0,1\}$ the set

[TABLE]

and for each pair $(m_{1},m_{2})\in\{0,1\}\times\{0,\ldots,\mathsf{W}_{2}-1\}$ the set

[TABLE]

Since the sets $\mathcal{C}_{0},\mathcal{C}_{1}$ cover $\mathcal{X}^{n}$ and since for each $\tilde{P}_{X}\in\mathcal{P}(\mathcal{X})$, it holds that \tilde{P}_{X}^{\otimes n}\big{[}X^{n}\in\mathcal{X}^{n}\big{]}=1, the subsets $\Gamma_{0,n},\Gamma_{1,n}$ cover the set $\mathcal{P}(\mathcal{X})$. Similarly, since for each $m_{1}\in\{0,1\}$ the sets $\mathcal{G}_{m_{1},0},\ldots,\mathcal{G}_{m_{1},\mathsf{W}_{2}-1}$ cover $\mathcal{Y}_{1}^{n}$, the subsets $\Delta^{n}_{m_{1},0},\ldots,\Delta^{n}_{m_{1},\mathsf{W}_{2}-1,n}$ cover the set $\mathcal{P}(\mathcal{Y}_{1})$. Moreover, by the constraint on the type-I error probability at Detectors $1$ and $2$, (147) and (148):

[TABLE]

By the union bound there exist thus an index $\tilde{m}_{1}\in\{0,1\}$ and an index pair $(m_{1}^{*},m_{2}^{*})\in\{0,1\}\times\{0,\ldots,\mathsf{W}_{2}-1\}$ such that:

[TABLE]

and

[TABLE]

Combining (162) with the definition of $\Delta_{\tilde{m}_{1},n}$ in (158) and with [24, Theorem 3] (recall that by assumption $P_{XY_{1}}(x,y_{1})>0$, for all $(x,y_{1})\in\mathcal{X}\times\mathcal{Y}_{1}$) yields that for any $\mu>0$ and sufficiently large $n$ :

[TABLE]

In the same way, combining (163) (158) with (159) and extending [24, Theorem 3] to three pmfs (recall that by assumption $P_{XY_{1}Y_{2}}(x,y_{1},y_{2})>0$, for all $(x,y_{1},y_{2})\in\mathcal{X}\times\mathcal{Y}_{1}\times\mathcal{Y}_{2}$), for sufficiently large $n$:

[TABLE]

Taking now $n\to\infty$ and $\mu\to 0$, by the continuity of the KL divergence we can conclude that if the exponent pair $(\theta_{1},\theta_{2})$ is achievable, then there exist subsets $\Gamma_{0},\Gamma_{1}$ that cover $\mathcal{P}(\mathcal{X})$, subsets $\Delta_{0,0},\ldots,\Delta_{0,\mathsf{W}_{2}-1}$ that cover $\mathcal{P}(\mathcal{Y}_{1})$, and subsets $\Delta_{1,0},\ldots,\Delta_{1,\mathsf{W}_{2}-1}$ that cover $\mathcal{P}(\mathcal{Y}_{1})$ so that:

[TABLE]

where the indices $b,c\in\{0,1\}$ and $c_{2}\in\{0,\ldots,\mathsf{W}_{2}-1\}$ are such that

[TABLE]

We continue to notice that the upper bounds in (164) become looser when elements are removed from the sets $\Gamma_{b}$, $\Gamma_{c}$, and $\Delta_{c,c_{2}}$. The converse statement thus remains valid by imposing

[TABLE]

Moreover, if $b=c$, then we impose

[TABLE]

and if $b\neq c$, then we impose that $\Gamma_{b}$ and $\Gamma_{c}$ form a partition.

If $b=c$, this concludes the proof. Otherwise, if $b\neq c$, we obtain the intermediate result that

[TABLE]

for two sets $\Gamma_{b}$ and $\Gamma_{c}$ forming a partition of $\mathcal{P}(\mathcal{X})$ and satisfying (165) and (166).

We now characterize the choice of the sets $\{\Gamma_{b},\Gamma_{c}\}$ that yields the loosest bound in (170). To this end, notice first that by assumption (146), constraints (170) are equivalent to:

[TABLE]

We notice that the right-hand side of (171) is upper bounded as:

[TABLE]

and that the bound holds with equality when

[TABLE]

This concludes the proof also for the case $b\neq c$.

Appendix C Proof of Converse Part of Theorem 7

Let $R_{2}=0$. Fix a rate $R_{1}\geq 0$ and a pair of exponents $(\theta_{1},\theta_{2})\in\mathcal{E}_{0}(R_{1},0)$. Then, choose an $\epsilon\in(0,1/2)$, a sufficiently large blocklength $n$, encoding and decision functions $\phi_{1,n}$, $\phi_{2,n}$, $\psi_{1,n}$, and $\psi_{2,n}$ that satisfy

[TABLE]

and

[TABLE]

Notice first that for each $i\in\{1,2\}$ [10]:

[TABLE]

where $h_{2}\left(p\right)$ denotes the entropy of a Bernouilli-$(p)$ memoryless source. Since $\alpha_{i,n}\leq\epsilon<1/2$, for each $i\in\{1,2\}$, Inequality (179) yields:

[TABLE]

with $\mu_{n}\triangleq\frac{1}{n(1-\epsilon)}h_{2}\left(\epsilon\right)$. Notice that $\mu_{n}\rightarrow 0$ as $n\rightarrow\infty$.

Consider now:

[TABLE]

where: $(a)$ follows by the data processing inequality for relative entropy; $(b)$ holds since $M_{1}$ and $Y^{n}_{1}$ are independent under the alternative hypothesis $\mathcal{H}=1$. $(c)$ is due to the chain rule for mutual information; $(d)$ follows since conditioning reduces entropy; $(e)$ is due to the Markov chain ${Y_{1}}^{k-1}-\!\!\!\minuso\!\!\!-(M_{1},X^{k-1})-\!\!\!\minuso\!\!\!-{Y_{1}}_{k}$; $(f)$ holds by defining $U_{k}\triangleq(M_{1},{X}^{k-1})$; $(g)$ is obtained by introducing a random variable $Q$ that is uniform over the set $\left\{1,\cdots,n\right\}$ and independent of all previously defined random variables; and $(h)$ holds by defining $U(n)\triangleq(U_{Q},Q)$ and $Y_{1}(n)\triangleq Y_{1Q}$.

In a similar way, one obtains:

[TABLE]

where $(i)$ follows by the data processing inequality for relative entropy; $(j)$ holds by the independence of the pair $(M,M_{2})$ with ${Y^{n}}_{2}$ under the alternative hypothesis $\mathcal{H}=1$; $(k)$ by the data processing inequality for relative entropy; $(\ell)$ holds since conditioning reduces entropy; $(o)$ follows by proceeding along the steps $(b)$ to $(h)$ above; and $(m)$ holds by defining $\tilde{\mu}_{n}\triangleq{\mathsf{W}_{2,n}}/(n(1-\epsilon))+\mu_{n}$.

Notice that by the assumption $R_{1}=0$, the term $1/n\log\|\mathsf{W}_{2,n}\|\to 0$ as $n\to\infty$. Thus, also $\tilde{\mu}_{n}\to 0$ as $n\to\infty$.

We next lower bound the rate $R$:

[TABLE]

For any blocklength $n$, the newly defined random variables $X(n),Y_{1}(n),Y_{2}(n)\sim P_{XY_{1}Y_{2}}$ and $U(n)-\!\!\!\minuso\!\!\!-X(n)-\!\!\!\minuso\!\!\!-(Y_{1}(n),Y_{2}(n))$. Letting now the blocklength $n\to\infty$, and then $\epsilon\to 0$, by continuity of mutual information establishes the desired converse result.

Appendix D Proof of Theorem 9

We analyze the scheme in Subsection V-B, where we focus on the type-II error probabilities. The analysis of the type-I error probabilities is standard and omitted.

To analyze the probability of type-II error at Detector 1, we notice that $\hat{\mathcal{H}}_{1}=0$ only if there exists an index $m_{1}\in\{1,\ldots,2^{nR_{1}}\}$ such that

[TABLE]

Therefore, using the union bound:

[TABLE]

where $x_{n}$ is a sequence that tends to [math] as $n\to\infty$. Inequality $(a)$ holds by Sanov’s theorem and by the way the source sequences and the codewords are generated and Inequality $(b)$ holds by the choice of $R_{1}$ in (78).

To analyze the probability of type-II error at Detector 2, we notice that $\hat{\mathcal{H}}_{2}=0$ only if there exists a pair of indices $(m_{1},m_{2})\in\{1,\ldots,2^{nR_{1}}\}\times\{1,\ldots,2^{nR_{2}}\}$ so that

[TABLE]

Therefore, applying similar steps as before:

[TABLE]

where $\xi_{n}^{\prime}$ is a sequence that tends to [math] as $n\to\infty$. The proof is then concluded by letting $n\to\infty$ and by noting that there must exist at least one pair of codebooks achieving the same exponents as the random ensemble.

Appendix E Proof of Theorem 11

We analyze the scheme in Subsection V-B. The type-II error probability at Detector 2 can be analyzed as in the preceding Appendix D.

To analyze the probability of type-II error at Detector 1, we notice that $\hat{\mathcal{H}}_{1}=0$ only if there exists an index $m_{1}\in\{1,\ldots,2^{nR_{1}}\}$ such that

[TABLE]

By now standard arguments:

[TABLE]

where the sequence $\xi_{n}^{\prime\prime}\to 0$ as $n\to\infty$.