First and Second Order Asymptotics in Covert Communication

TL;DR

This paper analyzes the fundamental limits of covert communication over binary-input channels, providing precise asymptotic characterizations under various covertness metrics and demonstrating the optimality of PPM schemes.

Contribution

It offers the first and second-order asymptotic characterizations for covert communication under multiple covertness metrics, clarifying the impact of metric choice on limits.

Findings

PPM achieves optimal first-order asymptotics across all metrics.

Exact second-order asymptotics are derived for relative entropy covertness.

The choice of covertness metric significantly influences the communication limits.

Abstract

We study the first- and second-order asymptotics of covert communication over binary-input DMC for three different covertness metrics and under maximum probability of error constraint. When covertness is measured in terms of the relative entropy between the channel output distributions induced with and without communication, we characterize the exact first- and second-order asymptotics of the number of bits that can be reliably transmitted with a maximum probability of error less than $\epsilon$ and a relative entropy less than $\delta$. When covertness is measured in terms of the variational distance between the channel output distributions or in terms of the probability of missed detection for fixed probability of false alarm, we establish the exact first-order asymptotics and bound the second-order asymptotics. PPM achieves the optimal first-order asymptotics for all three metrics, as well as the optimal second-order asymptotics for relative entropy. The main conceptual contribution of this paper is to clarify how the choice of a covertness metric impacts the information-theoretic limits of covert communications. The main technical contribution underlying our results is a detailed expurgation argument to show the existence of a code satisfying the reliability and covertness criteria.

Equations

$$\mathbb{D}\left(P\Vert Q\right)$$

$$\mathbb{V}\left(P,Q\right)$$

$${\chi }_2(P||Q)$$

$${\beta }_{\alpha }(P,Q)$$

$$P_Y(y)$$

$$I(P_X,W_{Y|X})$$

$$F_{XY|Q_Y}(\gamma )$$

$$\displaystyle{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}F_{XY|Q_{Y}X=x}(\gamma)}$$

$$F_{XY}(\gamma )$$

$${\overline{F}}_{XY}(\gamma )$$

$${\hat{P}}_{WXY}(w,x,y)\triangleq \frac{1}{M}\text{mathds}1\left\{x_w=x\right\}{W}_{Y|X}(y|x).$$

$$P_0\triangleq W_{Y|X=0},$$

$$Q_0\triangleq W_{Z|X=0},$$

$$f:[[1,K]]\times [[1,M]]\to {\mathcal{X}}^n\text{ and }\varphi :{\mathcal{Y}}^n\times [[1,K]]\to [[1,M]],$$

$${\hat{P}}_{\mathbf{Z}}(\text{z})\triangleq \sum _{s=1}^K\sum _{w=1}^M\frac{1}{MK}{W}_{Z|X}^{\text{scalebox}0.7\otimes }(\mathbf{z}|f(s,w)).$$

$$\displaystyle{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}P_{\textnormal{err}}^{*}\triangleq\max_{s\in\llbracket{1},{K}\rrbracket,w\in\llbracket{1},{M}\rrbracket}{\mathbb{P}}{\left(\phi(\mathbf{Y},s)\neq W|S=s,W=w\right)}\leqslant\epsilon.}$$

$$\mathbb{D}\left({\hat{P}}_{\mathbf{Z}}\Vert Q_0^{\text{scalebox}0.7\otimes }\right)⩽\delta ;$$

$$\mathbb{V}\left({\hat{P}}_{\mathbf{Z}},Q_0^{\text{scalebox}0.7\otimes }\right)⩽\delta ;$$

$${\beta }_{\alpha }(Q_0^{\text{scalebox}0.7\otimes },{\hat{P}}_{\mathbf{Z}})⩾1-\alpha -\delta .$$

$$M_D^{\ast }(n,ϵ,\delta )$$

$$\displaystyle{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\log M^{\text{*}}_{D}(n,\epsilon,\delta)}=\sqrt{\frac{2\delta}{\chi_{2}(Q_{1}\|Q_{0})}}D_{P}n^{\frac{1}{2}}-\sqrt{\sqrt{\frac{2\delta}{\chi_{2}(Q_{1}\|Q_{0})}}V_{P}}Q^{-1}(\epsilon)n^{\frac{1}{4}}+O(\log n),$$

$$D_P\triangleq \mathbb{D}\left(P_1\Vert P_0\right),$$

$$\log K=(1+\rho )\sqrt{\frac{2\delta }{{\chi }_2(Q_1\Vert Q_0)}}[D_Q-D_P{]}^{+}{n}^{\frac{1}{2}},$$

$$\underset{ϵ\to 0}{\lim }\underset{n\to \infty }{\lim }\frac{\log M_D^{\ast }(n,ϵ,\delta )}{\sqrt{n}}=\sqrt{\frac{2\delta }{{\chi }_2(Q_1\Vert Q_0)}}{D}_P.$$

$$\displaystyle{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\log M^{\text{*}}_{V}(n,\epsilon,\delta)}\leqslant\frac{2\Gamma D_{P}}{\sqrt{\chi_{2}(Q_{1}\|Q_{0})}}n^{\frac{1}{2}}-\sqrt{\frac{2\Gamma V_{P}}{\sqrt{\chi_{2}(Q_{1}\|Q_{0})}}}Q^{-1}(\epsilon)n^{\frac{1}{4}}+O(\log n),$$

$$\displaystyle{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\log M^{\text{*}}_{V}(n,\epsilon,\delta)}\geqslant\frac{2\Gamma D_{P}}{\sqrt{\chi_{2}(Q_{1}\|Q_{0})}}n^{\frac{1}{2}}-{\left(\sqrt{\frac{2\Gamma V_{P}}{\sqrt{\chi_{2}(Q_{1}\|Q_{0})}}}Q^{-1}(\epsilon)+\frac{2\sqrt{\pi}e^{\frac{\Gamma^{2}}{2}}D_{P}}{\sqrt{\Gamma}\chi_{2}(Q_{1}\|Q_{0})^{\frac{1}{4}}}\right)}n^{\frac{1}{4}}+O(\log n),$$

$$\log K=(1+\rho )\frac{2\Gamma D_P}{\sqrt{{\chi }_2(Q_1\Vert Q_0)}}[D_Q-D_P{]}^{+}{n}^{\frac{1}{2}},$$

$$\underset{ϵ\to 0}{\lim }\underset{n\to \infty }{\lim }\frac{\log M_V^{\ast }(n,ϵ,\delta )}{\sqrt{n}}=\frac{2\Gamma D_P}{\sqrt{{\chi }_2(Q_1\Vert Q_0)}}.$$

$$\displaystyle{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\log M^{\text{*}}_{\beta}(n,\epsilon,\delta,\alpha)}\leqslant\frac{(\Lambda+\Upsilon)D_{P}}{\sqrt{\chi_{2}(Q_{1}\|Q_{0})}}n^{\frac{1}{2}}-\sqrt{\frac{(\Lambda+\Upsilon)V_{P}}{\sqrt{\chi_{2}(Q_{1}\|Q_{0})}}}Q^{-1}(\epsilon)n^{\frac{1}{4}}+O(\log n),$$

$$\displaystyle{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\log M^{\text{*}}_{\beta}(n,\epsilon,\delta,\alpha)}\geqslant\frac{(\Lambda+\Upsilon)D_{P}}{\sqrt{\chi_{2}(Q_{1}\|Q_{0})}}n^{\frac{1}{2}}-{\left(\sqrt{\frac{(\Lambda+\Upsilon)V_{P}}{\sqrt{\chi_{2}(Q_{1}\|Q_{0})}}}Q^{-1}(\epsilon)+\frac{\sqrt{2\pi}{\left(e^{\frac{\Gamma^{2}}{2}}+e^{\frac{\Upsilon^{2}}{2}}\right)}D_{P}}{\sqrt{\Lambda+\Upsilon}\chi_{2}(Q_{1}\|Q_{0})^{\frac{1}{4}}}\right)}n^{\frac{1}{4}}+O(\log n).$$

$$\log K=(1+\rho )\frac{\left(\Lambda +{\rm Y}\right)[D_Q-D_P{]}^{+}}{\sqrt{{\chi }_2(Q_1\Vert Q_0)}}{n}^{\frac{1}{2}},$$

$$\underset{ϵ\to 0}{\lim }\underset{n\to \infty }{\lim }\frac{\log M_{\beta }^{\ast }(n,ϵ,\delta ,\alpha )}{\sqrt{n}}=\frac{(\Lambda +{\rm Y})D_P}{\sqrt{{\chi }_2(Q_1\Vert Q_0)}}.$$

$$\log \frac{W_{Y|X}(y|x_w)}{Q_Y(y)}>\gamma .$$

$${ϵ}_w^{(1)}$$

Full text

I Introduction

While most information-theoretic security works to date have revolved around the issues of confidentiality and authentication [2, 3, 4], the growing concern around mass communication surveillance programs has reignited interest in investigating the covertness of communications, also known as Low Probability of Detection (LPD). In LPD problems, the objective is to hide the presence of communication and not necessarily to prevent information leakage about the messages transmitted. Following the analysis of LPD with space-time codes [5], recent works have investigated the information-theoretic limits of covert communications over noisy channel [6, 7]. In particular, building upon concepts from steganography [8], the study in [6] shows the existence of a “square-root law” for covert communication, which essentially states that no more than $O(\sqrt{n})$ bits can be communicated covertly over $n$ channel uses of a memoryless channel.

The square-root law of covert communication has been refined in several follow-up works [9]; in particular, the information-theoretic limits are now known for classical discrete and Gaussian memoryless channels [10, 11], classical-quantum channels [12, 13], and multiple-access channels [14], when covertness is measured in terms of the relative entropy between the channel output distributions induced with and without communication. Note that the choice of relative entropy as a metric for covertness is guided in part by the natural connection between relative entropy and information-theoretic metrics, such as entropy and mutual information, which have been largely explored in the context of information-theoretic security [15].

The contribution of the present paper is twofold. First, as an attempt to develop operational characterizations of covertness, we study the information-theoretic limits of covert communication for alternative metrics, including variational distance, and probability of missed detection. Second, motivated by the likely time-limited nature of covert communications, we make a first step towards a finite-length analysis and extend the first-order analysis of information-theoretic limits to second-order asymptotics. The specific results developed in the present paper focus on binary-input DMCs to yield simple closed-form expressions and are the following.

• •

We characterize the exact second-order asymptotics of the maximum number of reliable and covert bits that can be transmitted with maximum probability of error $\epsilon$ and relative entropy $\delta$ between channel output distributions with and without communication (Theorem 1); this corrects an unfortunate error in the conference version [1], in which we claimed erroneous second-order asymptotics for arbitrary codes.

• •

We characterize the exact first-order and bound the second-order asymptotics of the maximum number of reliable and covert bits that can be transmitted with maximum probability of error $\epsilon$ and variational distance $\delta$ between channel output distributions with and without communication (Theorem 2).

• •

Finally, we characterize the exact first-order and bound the second-order asymptotics of the maximum number of reliable and covert bits that can be transmitted with maximum probability of error $\epsilon$ and probability of missed detection $1-\alpha-\delta$ when $\alpha$ is the adversary’s probability of false alarm (Theorem 3).

All our achievability results are established using Pulse-Position Modulation (PPM), which optimality was previously only established for the first-order asymptotics with relative entropy [16]. The operational relevance of codes used in conjunction with PPM, which may be viewed as a highly structured subset of constant composition codes, is justified by recent work towards practical code design [17], in which PPM plays a crucial role. We also emphasize from the outset that the focus on maximal probability of error is essential to our analysis. As discussed in Remark 1 and Appendix A, there is no strong converse for average probability of error without additional constraint; hence, considering the maximum probability of error as the reliability metric is reasonable for second-order analysis.

The second-order asymptotics obtained with a relative entropy metric for covertness are what could have been expected by extrapolating the results of channel coding in the finite length regime [18] to the first-order asymptotics of covert communication [10, 11]; however, the proof requires specific techniques beyond those used to study the first-order asymptotics of covert communications and second-order asymptotics of classical communication. First, as already mentioned, the achievability proof relies on PPM codes [16] instead of independent and identically distributed (i.i.d.) random codes. Second, unlike the analysis of the second-order asymptotics of reliable and secure communications [18, 19], we have to deal here with parameters capturing reliability and covertness, with the latter appearing in the first-order asymptotics; more specifically, the optimal coding scheme identified in [10] exploits a code with a bin structure, in which each bin forms a reliability code for the legitimate channel indexed by the secret key while the overall code forms a resolvability code for the adversary’s channel. To prove the existence of a code with the desired characteristics, we expurgate a random code after resorting to concentration of measure inequalities such as McDiarmid’s Inequality and carefully analyzing the probability of error. We point out that our current results only identify the second-order asymptotics for the number of transmitted message bits and do not characterize the second-order asymptotics for the number of key bits.

The remainder of the paper is organized as follows. In Section II, we formally introduce the model of covert communication and state our main results. In Section III, we develop a series of metric-independent results that form the basis of our analysis of covert communication. In Section IV-A, Section IV-B, and Section IV-C, we exploit the results of Section III to study a relative entropy metric, variational distance metric, and probability of missed-detection metric for covertness, respectively. In Section V, we conclude the paper with a discussion of possible further extensions and improvements.

II Model and main results

II-A Notation

inline]We need to define all important quantities, say D, V, $\beta$, base of log, notation for random variables, and distributions.

Throughout the paper, $\log$ and $\exp$ should be understood in base $e$. Moreover, random variables are denoted with upper case letters, e.g. $X$, while their realizations are denoted with lower case letters, e.g., $x$. The distribution of a random variable such as $X$ is denoted by $P_{X}$. Calligraphic letters are used for sets, e.g., $\mathcal{X}$, and boldface fonts are used for vectors e.g., x. For two integers $a$ and $b$, if $a\leqslant b$, we define $\llbracket{a},{b}\rrbracket\triangleq\{a,a+1,\cdots,b-1,b\}$; otherwise $\llbracket{a},{b}\rrbracket\triangleq\emptyset$. For any distribution $P$ over $\mathcal{X}$, P^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n} denotes the product distribution over $\mathcal{X}^{n}$, i.e., P^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}(\textbf{x})\triangleq\prod_{i=1}^{n}P(x_{i}). For two distributions $P$ and $Q$ over the same set $\mathcal{X}$, we define

[TABLE]

The notation $P\ll Q$ means that $P$ is absolutely continuous with respect to (w.r.t.) $Q$, i.e., if $Q(x)=0$ for some $x\in\mathcal{X}$ then $P(x)=0$. A DMC $(\mathcal{X},W_{Y|X},\mathcal{Y})$ consists of a finite input alphabet $\mathcal{X}$, a finite output alphabet $\mathcal{Y}$, and a transition probability $W_{Y|X}$ such that $W_{Y|X}(y|x)$ indicates the probability of obtaining $y$ at the output given that $x$ is transmitted at the input. For a DMC $(\mathcal{X},W_{Y|X},\mathcal{Y})$ and two distributions $P_{X}$ and $Q_{Y}$ on $\mathcal{X}$ and $\mathcal{Y}$, we define

[TABLE]

and for any $\gamma$,

[TABLE]

Moreover, given codewords $\{x_{i}\}_{i=1}^{M}\in{\mathcal{X}}^{M}$ and a uniform random variable $W\in\llbracket{1},{M}\rrbracket$, $\widehat{P}_{WXY}$ denotes the joint distribution induced on $(W,X,Y)$, i.e.,

[TABLE]

For any discrete random variable $A$, let $\mu_{A}\triangleq\min_{a:{\mathbb{P}}{\left(A=a\right)}>0}{\mathbb{P}}{\left(A=a\right)}$. For a real number $x$, we also define the $Q$-function $Q(x)\triangleq\int_{x}^{\infty}\frac{1}{\sqrt{2\pi}}e^{-\frac{x^{2}}{2}}\text{d}x$ and $[x]^{+}\triangleq\max(0,x)$. Moreover, for sequences $\{a_{n}\}$, $\{b_{n}\}$, and $\{c_{n}\}$, we have $a_{n}=b_{n}+O(c_{n})$ if and only if there exists a constant $K$ independent of $n$ such that $|a_{n}-b_{n}|\leqslant Kc_{n}$ for all $n$. Finally, for $\textbf{x}\in\mathcal{\{}0,1\}^{n}$, $\textnormal{wt}(\textbf{x})\triangleq\left|{\{i\in\llbracket{1},{n}\rrbracket:x_{i}=1\}}\right|$ is the weight of $\mathbf{x}$.

II-B Model and main results

We consider the situation illustrated in Fig. 1, in which a legitimate transmitter communicates with a legitimate receiver over a DMC $({\mathcal{X}},W_{Y|X},{\mathcal{Y}})$ in the presence of an adversary who observes communication through another DMC $({\mathcal{X}},W_{Z|X},{\mathcal{Z}})$. There is no loss in generality in ignoring the joint channel $W_{YZ|X}$, as we shall see that results only depend on the marginal channels $W_{Y|X}$ and $W_{Z|X}$. Furthermore, to obtain simple closed-form expressions, we assume in the sequel that ${\mathcal{X}}=\{0,1\}$, where [math] is an “innocent symbol” corresponding to the expected input of the channel if no communication were taking place. The extension beyond binary-inputs is briefly outlined in Section V. We also denote the output distribution induced by each input symbol by

[TABLE]

where it is assumed that $Q_{1}\ll Q_{0}$, $P_{1}\ll P_{0}$ and $Q_{1}\neq Q_{0}$. These assumptions are necessary for our results to hold. In fact, without $Q_{1}\ll Q_{0}$, covert communication is impossible as some input symbols detect the use of the non-innocent symbol with probability one. At the other extreme, if $Q_{1}=Q_{0}$ then no detector can distinguish the use of innocent and non-innocent symbols so that the problem reduces to classical communication. Finally, without $P_{1}\ll P_{0}$, Alice and Bob have an unfair advantage that allows them to exchange $\omega(\sqrt{n})$ bits covertly [10, Theorem 7].

The objective is for the transmitter to communicate a uniformly distributed message $W\in\llbracket{1},{M}\rrbracket$ with a small probability of error while ensuring low probability of detection from the adversary. The communication may be assisted by a uniformly distributed shared secret key $S\in\llbracket{1},{K}\rrbracket$. A formal description of a code is as follows.

Definition 1**.**

A code of blocklength $n$ consists of an encoder/decoder pair

[TABLE]

which induces the distribution $\widehat{P}_{\mathbf{Z}}$ at the adversary’s channel output, defined for $\mathbf{z}\in{\mathcal{Z}}^{n}$ as

[TABLE]

A code is $\epsilon$-reliable, if the pair $(f,\phi)$ is such that

[TABLE]

Moreover, an $\epsilon$-reliable code defined by $(f,\phi)$ is:

• •

an $(M,K,n,\epsilon,\delta)_{D}$ code, if it satisfies

[TABLE]

• •

an $(M,K,n,\epsilon,\delta)_{V}$ code, if it satisfies

[TABLE]

• •

an $(M,K,n,\epsilon,\delta,\alpha)_{\beta}$ code, if if it satisfies

[TABLE]

The maximum number of messages that can be transmitted by an $(M,K,n,\epsilon,\delta)_{D}$, $(M,K,n,\epsilon,\delta)_{V}$, and $(M,K,n,\epsilon,\delta,\alpha)_{\beta}$ code is denoted by $M^{*}_{D}(n,\epsilon,\delta)$, $M^{*}_{V}(n,\epsilon,\delta)$, and $M^{*}_{\beta}(n,\epsilon,\delta,\alpha)$, respectively.

Remark 1**.**

The definition of the probability of error in (16) differs from previous studies [11, 10]; we ask that the maximum probability of error be small for any choice of the key $S$ and message $W$. This more stringent condition captures a pragmatic requirement and is critical to our converse argument. As shown in Appendix A, analyzing the average probability of error is a different and (we believe) substantially more difficult endeavor since there is no strong converse for the reliability parameter. This intriguing behavior appears because one can slightly increase the average probability of error without any effect on covertness by simply adding all-zero codewords to the codebook. This of course leads to what one would consider as “bad codes,” which we avoid by focusing on the maximal probability of error.

Remark 2**.**

The three covertness metrics in (17)-(19) have slightly different operational meanings. When enforcing that the probability of missed detection satisfy \beta_{\alpha}(Q_{0}^{{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}},\widehat{P}_{\mathbf{Z}})\geqslant 1-\alpha-\delta for a fixed probability of false alarm $\alpha$, one implicitly assumes that the adversary optimizes its detector at a specific point of the Receiver Operation Characteristic (ROC) curve with known probability of false alarm. In contrast, when enforcing {{\mathbb{V}}\!\left(\smash{\widehat{P}_{\mathbf{Z}}},Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}\right)}\leqslant\delta, since \alpha+\beta_{\alpha}(Q_{0}^{{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}},\widehat{P}_{\mathbf{Z}})\geqslant 1-{{\mathbb{V}}\!\left(\smash{\widehat{P}_{\mathbf{Z}}},Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}\right)} for any probability of false alarm $\alpha$, one essentially wishes to enforce covertness irrespective of the exact operating point on the adversary’s ROC curve. Finally, since {{\mathbb{V}}\!\left(\smash{\widehat{P}_{\mathbf{Z}}},Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}\right)}^{2}\leqslant\frac{1}{2}{{{\mathbb{D}}\!\left({\smash{\widehat{P}_{\mathbf{Z}}}\|Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}}\right)}} by Pinsker’s Inequality, the constraint {{\mathbb{D}}\!\left({\smash{\widehat{P}_{\mathbf{Z}}}\|Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}}\right)}\leqslant\delta is more stringent than when using variational distance, but only provides a loose proxy for constraining the ROC curve since Pinsker’s inequality is not tight..111See also the discussion in [10, Appendix A]. The relations between the covertness metrics immediately lead to the following ordering of the maximum number of covert bits.

[TABLE]

Our main results in Theorem 1, Theorem 2, and Theorem 3 characterize the maximum number of reliable and covert bits defined above as a function of the channel characteristics and the blocklength $n$. In all cases, the first and second terms behave as $\Theta\left(\smash{n^{\frac{1}{2}}}\right)$ and $\Theta\left(\smash{n^{\frac{1}{4}}}\right)$, respectively, as expected; nevertheless, the constant behind $\Theta(\cdot)$ is metric-specific.

Theorem 1**.**

For $\epsilon\in]0,1[$ and $\delta>0$, we have

[TABLE]

with

[TABLE]

This optimal number of message bits is obtained with a first-order optimal number of key bits

[TABLE]

where $\rho>0$ can be arbitrarily small and $D_{Q}={{\mathbb{D}}\!\left({Q_{1}\|Q_{0}}\right)}$. In addition,

[TABLE]

Theorem 2**.**

For $\epsilon\in]0,1[$ and $\delta\in]0,1[$, let \Gamma\triangleq Q^{{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}-1}}{\left(\frac{1-\delta}{2}\right)}. We have

[TABLE]

and

[TABLE]

where $D_{P}$ and $V_{P}$ are as in (22). This number of message bits is obtained with a number of key bitsinline]Is the number of key bits optimal?

[TABLE]

where $\rho>0$ can be arbitrarily small. In addition,

[TABLE]

Theorem 3**.**

For $\epsilon\in]0,1[$, $\alpha\in]0,1[$, and $\delta\in]0,1-\alpha[$, let $\Lambda\triangleq Q^{-1}(1-\alpha-\delta)$ and $\Upsilon\triangleq Q^{-1}(\alpha)$. We have

[TABLE]

and

[TABLE]

This number of message bits is obtained with a number of key bitsinline]Is the number of key bits optimal?

[TABLE]

where $\rho>0$ can be arbitrarily small. In addition,

[TABLE]

We point out that the second order asymptotics in Theorem 2 and Theorem 3 are loose because the parameters $\Gamma$, $\Lambda$, and $\Upsilon$ may be very small; in particular, because of the inequalities (20), Theorem 1 is sometimes a tighter lower bound for small values of $n$. We conjecture that the upper-bounds in (25) and (29) can be achieved, although we could not establish it with our current proof techniques.

We illustrate the results of the three theorems with a simple numerical example. We consider the situation in which $({\mathcal{X}},W_{Y|X},{\mathcal{Y}})$ and $({\mathcal{X}},W_{Z|X},{\mathcal{Z}})$ are Binary Symmetric Channels with cross-over probability $p_{m}=0.11$ and $p_{w}=0.45$, respectively, and $\epsilon=10^{-3}$, $\delta=10^{-2}$, and $\alpha=0.2$. Since the convergence to the asymptotic limit is slow (on the order of $\Theta(n^{-\frac{1}{4}})$), we use a log scale for the blocklength. As shown in Fig. 2, the choice of the covertness metric results in different number of bits, which of course raises the question of which number to settle on. We argue that $M^{*}_{V}(n,\epsilon,\delta)$ is the number to focus on since total variation distance satisfies two desirable properties: it is directly related to the performance of the adversary’s detector through the inequality \alpha+\beta\geqslant 1-{{\mathbb{V}}\!\left(\smash{\widehat{P}_{\mathbf{Z}}},Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}\right)} and it does not presume any knowledge about the exact operating point on the adversary’s ROC curve.

Before we detail the achievability and converse proofs in the next sections, we provide here a high-level sketch of the proofs. Following [10], the coding scheme in the achievability proof consists of $MK$ randomly generated codewords $\mathbf{x}_{sw}$ with $s\in\llbracket{1},{K}\rrbracket$ and $w\in\llbracket{1},{M}\rrbracket$. The code is designed such that the following two properties hold:

• (P1)

any “large” enough subset of codebook $\{\mathbf{x}_{sw}\}_{(s,w)\in\llbracket{1},{K}\rrbracket\times\llbracket{1},{M}\rrbracket}$ forms a resolvability code for the channel $({\mathcal{X}},W_{Z|X},{\mathcal{Z}})$ approximating Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n};

• (P2)

for every $s\in\llbracket{1},{K}\rrbracket$, the sub-codebook $\{\mathbf{x}_{sw}\}_{w\in\llbracket{1},{M}\rrbracket}$ forms a reliability code for the channel $({\mathcal{X}},W_{Y|X},{\mathcal{Y}})$ with a maximum probability of error $\epsilon$.

The analysis does not follow from standard arguments for two reasons. First, we resort to PPM codes [16] to obtain the optimal second-order asymptotics for relative entropy. Second, one cannot ensure (P1) and (P2) by merely expurgating some of the sub-codebooks, since this expurgation may change the output distribution on ${\mathcal{Z}}^{n}$ induced by the code; we address this by carefully analyzing the probability of error and using concentration of measure results. The converse proof adapts arguments from [18] after showing how the covertness constraint leads to an upper bound for the average codeword weight.

III Covert communications with generic covertness quasi-metric

We now develop results for a generic covertness metric, which will be specialized to the three metrics highlighted in Definition 1 in Section IV-A, Section IV-B, and Section IV-C. This organization allows us to separately handle the part of the analysis that solely depends on the code structure and not on the exact metric choice. Specifically, throughout this section, we consider an arbitrary quasi-metric $d$ measuring the distance between distributions. The quasi-metric $d$ does not necessarily possess the standard characteristics of a metric; however, we assume that it is non-negative and satisfies a modified triangle inequality, i.e.,

• •

$\forall P,Q:\quad d(P,Q)\geqslant 0$

• •

$\forall P,Q,R:\quad d(R,Q)\leqslant d(P,Q)+{{\mathbb{D}}\!\left({R\|P}\right)}+\sqrt{{{\mathbb{D}}\!\left({R\|P}\right)}}\max\left(1,\log\frac{1}{\min_{x:Q(x)>0}Q(x)}\right)$.

This modified version of the triangle inequality is intimately linked to the relative entropy, and is used mainly for convenience; this will avoid repetitions when we study different metrics and should not be given much operational significance. In brief, we will use the modified triangle inequality as follows: we will pick an appropriate distribution $P$ such that $d(R,Q)$ is essentially on the order of $d(P,Q)$ that depends on $d$, while the remaining terms that depend on relative entropy will be made negligible. We define an $(M,K,n,\epsilon,\delta)_{d}$ covert code and $M^{*}_{d}(n,\epsilon,\delta)$ for a covert communication channel $({\mathcal{X}},W_{Y|X},W_{Z|X},{\mathcal{Y}},{\mathcal{Z}})$ as in Definition 1.

Our proof for a generic covertness quasi-metric $d$ is organized as follows. In Section III-A, we start by developing one-shot results for random codes to bound the average probability of error (Lemma 1) and the approximation of output statistics as measured with the quasi-metric (Lemma 2 and Lemma 3). The expurgation of the code requires some care because of the interplay between reliability and channel resolvability (Lemma 4). In Section III-B, we specialize the one-shot results to obtain an achievability result for random codes with large-enough blocklength (Theorem 5). In Section III-C, we refine the statement of Theorem 5 for the specific random code ensemble consisting of PPM codes. This requires the study of the specific quantities involved in the statement of Theorem 5 (Lemma 5, Lemma 6, Lemma 7). We also pave the way for our final results by developing bounds between the output distribution induced by a PPM ensemble and the innocent distribution for relative entropy, variational distance, and probability of missed-detection (Lemma 8). These bounds are fairly technical and rely on moment bounds (Lemma 16). Finally, in Section III-D, we develop a general converse (Theorem 7).

III-A One-shot achievability analysis

We develop one-shot random coding results for codes simultaneously ensuring channel reliability and channel resolvability. Given a DMC$(\mathcal{X},W_{Y|X},\mathcal{Y})$ and a codebook with $M$ codewords $x_{1},\cdots,x_{M}\in\mathcal{X}$,222 Since we are deriving one-shot bounds, ${\mathcal{X}}$ can be arbitrary in this section. our analysis of reliability is based on a threshold decoder [18] operating as follows. Given $\gamma>0$ and a distribution $Q_{Y}$ on $\mathcal{Y}$, and upon observing $y$, the decoder forms the estimate $\widehat{W}=w$ of the transmitted message $W$ if there exists a unique $w\in\llbracket{1},{M}\rrbracket$ such that

[TABLE]

If there is no $w$ satisfying (33), the decoder declares an error. It is known [18] that the conditional probability of error when $W=w$ is upper bounded by $\epsilon_{w}^{(1)}+\epsilon_{w}^{(2)}$ where

[TABLE]

Under random coding, $\epsilon_{1}^{(1)},\cdots,\epsilon_{w}^{(1)}$ are independent, and we can therefore bound their average using well-known concentration inequalities. The following lemma upper bounds the expectation of $\epsilon_{w}^{(1)}$ and $\epsilon_{w}^{(2)}$.

Lemma 1**.**

Let $(\mathcal{X},W_{Y|X},\mathcal{Y})$ be a DMC and $P_{X}$ and $Q_{Y}$ be two distributions on $\mathcal{X}$ and $\mathcal{Y}$. Assume that $P_{Y}\ll Q_{Y}$. If we choose $X_{1},\cdots,X_{M}$ independently according to $P_{X}$, we have for all $\gamma\in\mathbb{R}$ and $w\in\llbracket{1},{M}\rrbracket$

[TABLE]

Proof:

The proof is similar to the proof of [10, Lemma 3] or [18, Theorem 18], but for completeness we provide the short proof. We know that

[TABLE]

Moreover, we have

[TABLE]

∎

Remark 3**.**

In a second-order analysis, $\gamma$ is generally chosen such that ${\mathbb{E}}{\left(\epsilon_{w}^{(1)}\right)}$ dominates ${\mathbb{E}}{\left(\epsilon_{w}^{(2)}\right)}$; we follow this approach as well, which is convenient since we have some flexibility in the bounding of ${\mathbb{E}}{\left(\epsilon_{w}^{(1)}\right)}$.

Remark 4**.**

The extra factor ${\mathbb{E}_{P_{Y}}}{\left(\frac{P_{Y}(Y)}{Q_{Y}(Y)}\right)}$ in (37), which does not appear in [18, Theorem 18], is the penalty of using $Q_{Y}$ instead of the “true” probability distribution of $Y$, $P_{Y}$, in the definition of $F_{XY|Q_{Y}}$. This sub-optimal bound yields the same first- and second-order asymptotics for covert communications, although it could certainly be improved for a finite-length analysis; this choice , however, simplifies our calculations.

Next, we develop one-shot channel resolvability results. In our proofs, we treat the distance between induced output distribution and the desired distribution as a function of independently generated codewords, which allows us to prove a super-exponential concentration inequality in Lemma 2 using McDiarmid’s Theorem. This may be viewed as an alternative and more direct approach to [20] and we recall this concentration inequality below for convenience.

Theorem 4** (McDiarmid’s Theorem).**

Let $\textbf{X}\triangleq(X_{1},\cdots,X_{n})$ be a sequence of independent random variables defined on $\mathcal{X}$. Furthermore, suppose $g:\mathcal{X}^{n}\to\mathbb{R}$ is a function satisfying

[TABLE]

Then, for all $\lambda>0$, we have

[TABLE]

Proof:

See [21, Theorem 2.2.2]. ∎

Lemma 2**.**

Consider a DMC $(\mathcal{X},W_{Z|X},\mathcal{Z})$ and a distribution $P_{X}$ on $\mathcal{X}$. If $\{x_{w}\}_{w=1}^{M}\in\mathcal{X}^{M}$ are $M\geqslant 2$ codewords333 Unlike the main problem in which we have $MK$ codewords, we denote here the total number of codewords by $M$ for simplicity. and $\widehat{P}_{Z}(z){\triangleq\frac{1}{M}\sum_{w=1}^{M}W_{Z|X}(x|x_{w})}$ is the corresponding induced distribution on $\mathcal{Z}$, we define the functions $g_{1},g_{2}:\mathcal{X}^{M}\to\mathbb{R}$ as

[TABLE]

and

[TABLE]

with $P_{Z}(z)=\sum_{x}P_{X}(x)W_{Z|X}(z|x)$. Then, for all $x_{1},\cdots,x_{M},x_{i}^{\prime}\in\mathcal{X}$, we have

[TABLE]

and

[TABLE]

Moreover, if $X_{1},\cdots,X_{M}$ are i.i.d. with distribution $P_{X}$, then

[TABLE]

and

[TABLE]

Proof:

We assume that $\widehat{P}_{Z}^{1}$ and $\widehat{P}_{Z}^{2}$ are the distributions induced by the codebooks $\mathcal{C}_{1}\triangleq\{x_{1},\cdots,x_{i},\cdots,x_{M}\}$ and $\mathcal{C}_{2}\triangleq\{x_{1},\cdots,x_{i}^{\prime},\cdots,x_{M}\}$, respectively. First, note that

[TABLE]

Next, for any two distributions $P$ and $Q$, by [22, Lemma 2.7], we have

[TABLE]

Therefore, we have

[TABLE]

where $(a)$ follows from the fact that $x\log(\left|\mathcal{Z}\right|/x)$ is increasing for $0<x<\left|\mathcal{Z}\right|/e$. Accordingly, we obtain

[TABLE]

Finally, using Theorem 4, we obtain (55) and (56). ∎

The following lemma bounds the expected value of the relative entropy appearing in Lemma 2. The result has appeared in the case of i.i.d. distributions in [15] and we provide here a one-shot result better-suited for covert communications and non-i.i.d. distributions.

Lemma 3**.**

Let $(\mathcal{X},W_{Z|X},\mathcal{Z})$ be a DMC and $P_{X}$ be a distribution on $\mathcal{X}$. If $X_{1},\cdots,X_{M}$ are i.i.d. with distribution $P_{X}$, and \widehat{P}_{Z}(z){\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}\triangleq\frac{1}{M}\sum_{w=1}^{M}W_{Z|X}(z,X_{w})} is the corresponding induced distribution on $\mathcal{Z}$, we have

[TABLE]

for all $\gamma\in\mathbb{R}$ where $P_{Z}(z)=\sum_{x}P_{X}(x)W_{Z|X}(z|x)$.

Proof:

From [23, Equation 10], we know that

[TABLE]

If we define the event $\mathcal{E}\triangleq\left\{(x,z):\log\frac{W_{Z|X}(z|x))}{P_{Z}(z)}\leqslant\gamma\right\}$, then we get

[TABLE]

∎

Consider now the DMC $({\mathcal{X}},W_{Y|X},W_{Z|X},{\mathcal{Y}},{\mathcal{Z}})$ and a randomly generated code $\{X_{sw}:s\in\llbracket{1},{K}\rrbracket,w\in\llbracket{1},{M}\rrbracket\}$. The next lemma leverages Lemma 1 and Lemma 2 to guarantee that, with positive probability, the entire code is a channel resolvability code for the channel $W_{Z|X}$ and, simultaneously, the subcode corresponding to every $s\in\llbracket{1},{K}\rrbracket$ is a channel reliability code for the channel $W_{Y|X}$. Note that a code with these properties is not necessarily a covert code, but we show in the next sections how a careful choice of the random coding distribution leads to covertness.

Lemma 4**.**

Consider a DMC $(\mathcal{X},W_{Y|X},W_{Z|X},\mathcal{Y},\mathcal{Z})$ and two probability distributions $P_{X}$ and $Q_{Y}$ on $\mathcal{X}$ and $\mathcal{Y}$. We sample codewords $\{X_{sw}:s\in\llbracket{1},{K}\rrbracket,w\in\llbracket{1},{M}\rrbracket\}$ independently according to $P_{X}$. For $s\in\llbracket{1},{K}\rrbracket$ and $w\in\llbracket{1},{M}\rrbracket$, let $\epsilon_{sw}^{(1)}$ and $\epsilon_{sw}^{(2)}$ be defined similar to (34) and (35). Moreover, for a fixed subset ${\mathcal{I}}\subset\llbracket{1},{K}\rrbracket\times\llbracket{1},{M}\rrbracket$, we define the induced distribution by the codewords in ${\mathcal{I}}$ as $\widehat{P}^{{\mathcal{I}}}_{\mathbf{Z}}(z)\triangleq\frac{1}{|{\mathcal{I}}|}\sum_{(s,w)\in{\mathcal{I}}}W_{Z|X}(z|X_{sw})$. The quantities $\epsilon_{sw}^{(1)}$, $\epsilon_{sw}^{(2)}$, and ${{\mathbb{D}}\!\left({\widehat{P}_{Z}^{{\mathcal{I}}}\|P_{Z}}\right)}$ will be treated as random variables with respect to the randomness of the random code. Finally, for every $\lambda_{1}$, $\lambda_{2}$, $\lambda_{3}$, $\gamma_{1}$, and $\gamma_{2}>0$, we define the events

[TABLE]

and

[TABLE]

Then ${\mathbb{P}}{\left(\mathcal{E}_{1}\cap\mathcal{E}_{2}\right)}>0$ if

[TABLE]

Proof:

We analyze the probability of each event separately. First, note that

[TABLE]

where $(a)$ follows since by (34), $\max_{sw}\epsilon^{(1)}_{sw}\leqslant\max_{x\in\textnormal{supp}P_{X}}F_{XY|Q_{Y}X=x}(\gamma_{1})$ holds almost surely, $(b)$ follows since all sub-codebooks are generated independently, and $(c)$ follows from Markov inequality and Lemma 1. Furthermore, we have

[TABLE]

where $(a)$ follows from Lemma 3, $(b)$ follows from Lemma 2, and $(c)$ follows from $\binom{MK}{\lambda_{1}MK}\leqslant\exp\left(MK{\mathbb{H}_{b}}\left(\lambda_{1}\right)\right)$. Combining these inequalities, we get

[TABLE]

Thus, we obtain

[TABLE]

where $(a)$ follows from (89). Finally, (79) ensures that the RHS of (93) is positive. ∎

III-B Asymptotic results for covert communications

We specialize the one-shot result of Lemma 4 to show the existence of covert codes for large blocklength $n$.

Theorem 5**.**

Consider a sequence of distributions $\{P_{\textbf{X}}^{n}\}_{n\geqslant 1}$ where $P_{\textbf{X}}^{n}$ is defined on ${\mathcal{X}}^{n}$ and two sequences of natural numbers $\{M_{n}\}_{n\geqslant 1}$ and $\{K_{n}\}_{n\geqslant 1}$. For a quasi-metric $d$ and $n$ large enough, if we have

[TABLE]

[TABLE]

[TABLE]

and

[TABLE]

then $M_{d}^{*}(n,\epsilon,\delta)\geqslant M_{n}$.

Theorem 5 is useful in that it identifies sufficient conditions for the existence of codes. In particular, subsequent achievability proofs are reduced to choosing an appropriate sequence of random coding distributions $\{P_{\textbf{X}}^{n}\}_{n\geqslant 1}$ and verifying inequalities (94)-(97). We will use a PPM distribution, which we introduce and analyze precisely in Section III-C.

Proof:

For a fixed $n$, we choose codewords $\textbf{X}_{sw}$ for $s\in\llbracket{1},{K_{n}}\rrbracket$ and $w\in\llbracket{1},{M_{n}}\rrbracket$ independently according to $P_{\textbf{X}}^{n}$. In Lemma 4, if we set

[TABLE]

then (94) implies (79) for large $n$. Therefore, we have with positive probability for $n$ large enough,

[TABLE]

where (99) follows from (34) with Q_{Y}=P_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}. Also, for all ${\mathcal{I}}\subset\llbracket{1},{K_{n}}\rrbracket\times\llbracket{1},{M_{n}}\rrbracket$ with $|{\mathcal{I}}|=\lambda_{1}M_{n}K_{n}$ and with $\mu_{\mathbf{Z}}\triangleq\min_{\textbf{z}}P_{\mathbf{Z}}(\textbf{z})$, we have for large enough $n$

[TABLE]

where $(a)$ follows from (96) and the choice of $\gamma_{2}$, $\lambda_{3}$. We next expurgate the $(1-\lambda_{1})M_{n}$ codewords with largest $\epsilon_{sw}^{(2)}$ for all $s$ from the sub-codebook $\{\mathbf{X}_{sw}:w\in\llbracket{1},{M_{n}}\rrbracket\}$ to form a new codebook ${\mathcal{I}}$ with $|{\mathcal{I}}|=\lambda_{1}M_{n}K_{n}$. For the new code, by Markov inequality, we have

[TABLE]

Hence, the maximum probability of error for the expurgated code is bounded by

[TABLE]

Finally, to prove the covertness of the code, note that for $n$ large enough

[TABLE]

where $(a)$ follows because $d$ is a quasi-metric, $(b)$ follows from (97), and $(c)$ follows from (102). ∎

III-C PPM distribution

We now study the specific “PPM distribution,” which was used in [16] for covert communication; we subsequently use this distribution to generate random codes and combine with Theorem 5. The choice of PPM is crucial to evaluate (95).

Definition 2** ($(n,\ell)$-PPM covert distribution).**

Given $\mathcal{X}=\{0,1\}$ and $n\geqslant\ell\geqslant 1$, we define the distribution $P_{\mathbf{X},\textnormal{PPM}}^{n,\ell}$ on $\mathcal{X}^{n}$ as follows. If $n=m\ell+r$ for $0\leqslant r<\ell$, we partition the set $\llbracket{1},{n}\rrbracket$ into $\ell$ sets of size $m$ and one set of size $r$. For simplicity, we consider the following partition: $\mathcal{B}_{i}=\{(i-1)m+1,\cdots,im\}$ for $i\in\llbracket{1},{\ell}\rrbracket$ and $\mathcal{B}_{\ell+1}=\{\ell m+1,\cdots,n\}$. Then, $P_{\mathbf{X},\textnormal{PPM}}^{n,\ell}$ is the uniform distribution on the sequences with exactly one “1” in all $\mathcal{B}_{i}$ for $i\in\llbracket{1},{\ell}\rrbracket$ and no “1” in $\mathcal{B}_{\ell+1}$, i.e.,

[TABLE]

We denote the output distribution of $P_{\mathbf{X},\textnormal{PPM}}^{n,\ell}$ for DMCs $(\mathcal{X},W_{Y|X},\mathcal{Y})$ and $(\mathcal{X},W_{Z|X},\mathcal{Z})$ by $P_{\mathbf{Y},\textnormal{PPM}}^{n,\ell}$ and $P_{\mathbf{Z},\textnormal{PPM}}^{n,\ell}$, respectively.

The PPM covert distribution $P_{\mathbf{X},\textnormal{PPM}}^{n,\ell}$ may be viewed as a way to generate structured constant composition codewords; specifically, every codeword generated according to $P_{\mathbf{X},\textnormal{PPM}}^{n,\ell}$ contains exactly $\ell$ 1-symbols, each located in a window of size $m$, the quotient of the Euclidean division of $n$ by $\ell$. The structure of the PPM covert distribution plays a pivotal role in our analysis.

We devote the remainder of this section to the development of properties of PPM distribution, which are geared towards the use of Theorem 5. We make repeated use of the Berry-Esseen Theorem, which we recall here for convenience.

Theorem 6** (Berry-Esseen Theorem).**

Let $X_{1},\cdots,X_{n}$ be independent random variables s.t. for $k\in\llbracket{1},{n}\rrbracket$ we have ${\mathbb{E}}{\left(X_{k}\right)}=\mu_{k}$, $\sigma_{k}^{2}={\text{{Var}}{\left(X_{k}\right)}}$, and $t_{k}={\mathbb{E}}{\left(|X_{k}-\mu_{k}|^{3}\right)}$. If we define \sigma^{2}=\sum_{{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}k}=1}^{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}n}\sigma_{k}^{2} and T=\sum_{k=1}^{n}t_{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}k}, then we have

[TABLE]

We now upper bound the left-hand-side (l.h.s.) of (95) for the PPM distribution in Lemma 5 and Lemma 6.

Lemma 5**.**

Let $(\mathcal{X},W_{Y|X},\mathcal{Y})$ be a binary-input DMC. If $(\mathbf{X},\mathbf{Y})$ is distributed according to P_{\mathbf{X},\textnormal{PPM}}^{n,\ell}W_{Y|X}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}, we have

[TABLE]

with

[TABLE]

Proof:

For a sequence $\mathbf{x}_{0}$ with $P_{\mathbf{X},\textnormal{PPM}}^{n,\ell}(\textbf{x}_{0})>0$, we can assume without loss of generality that the support of $\mathbf{x}_{0}$ is $\llbracket{1},{\ell}\rrbracket$ since the channel is memoryless and by relabeling. Hence, we have

[TABLE]

where $(a)$ follows from Theorem 6. ∎

Lemma 6**.**

For a binary-input DMC $(\mathcal{X},W_{Y|X},\mathcal{Y})$, we have

[TABLE]

Proof:

To prove (118), we define $m\triangleq\lfloor n/\ell\rfloor$. We first consider $P_{\mathbf{X},\textnormal{PPM}}^{m,1}$ for which we have

[TABLE]

Since $P_{\mathbf{Y},\textnormal{PPM}}^{n,\ell}$ is the product of the distributions over different blocks, i.e., P_{\mathbf{Y},\textnormal{PPM}}^{n,\ell}=\left(P_{\mathbf{Y},\textnormal{PPM}}^{m,1}\right)^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}\ell}\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}P}_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n\text{ mod }\ell}, upon defining $\mathbf{Y}_{i}=(Y_{(i-1)m+1},\cdots,Y_{im})$ for $i\in\llbracket{1},{\ell}\rrbracket$, we obtain

[TABLE]

∎

We now upper-bound the l.h.s. of (96) in Lemma 7.

Lemma 7**.**

Given a binary-input DMC $(\mathcal{X},W_{Z|X},\mathcal{Z})$, if $(\mathbf{X},\mathbf{Z})$ is distributed according to P_{\mathbf{X},\textnormal{PPM}}^{n,\ell}W_{Z|X}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}, then, for $\gamma\geqslant\ell D_{Q}$, we have

[TABLE]

Proof:

When $(\mathbf{X},\mathbf{Z})$ is distributed according to P_{\mathbf{X},\textnormal{PPM}}^{n,\ell}W_{Z|X}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}, for the random vector $\mathbf{Z}\in\mathcal{Z}^{n}$, the first $\ell$ blocks of length $m\triangleq\lfloor n/\ell\rfloor$ are denoted by $\mathbf{Z}_{1},\cdots,\mathbf{Z}_{\ell}$ with $\mathbf{Z}_{i}=(Z_{(i-1)m+1},\cdots,Z_{im})$. Moreover, $P_{\mathbf{Z}_{i}}$ denotes the distribution of block $\mathbf{Z}_{i}$. Therefore, we have

[TABLE]

where $(a)$ follows from Hoeffding’s Inequality with

[TABLE]

and I(P_{\mathbf{X},\textnormal{PPM}}^{m,1},W_{Z|X}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}m}) is defined as in (6). By [16, Lemma 1.5], we know that I(P_{\mathbf{X},\textnormal{PPM}}^{m,1},W_{Z|X}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}m})\leqslant D_{Q} and $B\leqslant\log m$ for $m$ large enough.444The bound for $B$ corrects a small oversight with no incidence on the result in [16]. Hence, we obtain

[TABLE]

∎

Finally, we upper-bound the l.h.s. of (97) using Lemma 8.

Lemma 8**.**

Let $({\mathcal{X}},W_{Z|X},{\mathcal{Z}})$ be a DMC, and $n$ and $\ell$ be positive integers with $m\triangleq\left\lfloor n/\ell\right\rfloor$ large enough and $\ell=\Theta(m)$. Then, we have

[TABLE]

Furthermore, we have

[TABLE]

Proof:

By [16, Lemma 1], we have for $n$ large enough and some positive constant $C_{1}$, $C_{2}$, $C_{3}$, and $C_{4}$,

[TABLE]

To prove (136), note that for any two distributions $P$ and $Q$, one can write ${{\mathbb{V}}\!\left(P,Q\right)}={\mathbb{P}_{P}}{\left(P(X)\geqslant Q(X)\right)}-{\mathbb{P}_{Q}}{\left(P(X)\geqslant Q(X)\right)}$. Therefore, if $\mathbf{Z}=(Z_{1},\cdots,Z_{n})$ and $\widetilde{\mathbf{Z}}=(\widetilde{Z}_{1},\cdots,\widetilde{Z}_{n})$ are distributed according to Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n} and $P_{\mathbf{Z},\textnormal{PPM}}^{n,\ell}$, respectively, then

[TABLE]

However, note that P_{\mathbf{Z},\textnormal{PPM}}^{n,\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}\ell}=(P_{\mathbf{Z},\textnormal{PPM}}^{m,1})^{\ell}\otimes Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}r}, and for any $\mathbf{z}=(z_{1},\cdots,z_{n})$ with $\mathbf{z}_{i}=(z_{(i-1)m+1},\cdots,z_{im})$, we have

[TABLE]

for $A(z)\triangleq\frac{Q_{1}(z)-Q_{0}(z)}{Q_{0}(z)}$. Thus, if we define

[TABLE]

Then, we have

[TABLE]

Furthermore, we consider two i.i.d. sequences $\underline{B}_{1},\cdots,\underline{B}_{\ell}$ and $\widetilde{\underline{B}}_{1},\cdots,\widetilde{\underline{B}}_{\ell}$ with distribution

[TABLE]

Note that with $\tau\triangleq{\mathbb{P}}{\left(A(Z_{1})=-1\right)}<1$, we have ${\mathbb{P}}{\left(B_{i}=-1\right)}=\tau^{m}$. By defining $\underline{C}_{i}\triangleq\log(1+\underline{B}_{i})$ and $\underline{\widetilde{C}}_{i}\triangleq\log(1+\widetilde{\underline{B}}_{i})$, we will deal with random variables that take finite values with probability one. We now replace $\widetilde{C}_{i}$ by $\underline{\widetilde{C}}_{i}$, and since $\underline{\widetilde{C}}_{1},\cdots\underline{\widetilde{C}}_{\ell}$ are i.i.d., we apply Theorem 6 to obtain

[TABLE]

where $(a)$ follows from Lemma 16 in Appendix B. Analogously, using Lemma 16 we obtain

[TABLE]

and therefore, we have (136).

To lower-bound \beta_{\alpha}(Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n},P_{\mathbf{Z},\textnormal{PPM}}^{n,\ell}), we use the Neyman-Pearson lemma, which states that if \alpha\leqslant{\mathbb{P}}{\left(\log\frac{P_{\mathbf{Z},\textnormal{PPM}}^{n,\ell}({\mathbf{Z}})}{Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}({\mathbf{Z}})}\geqslant\gamma\right)} for some $\gamma$, then we have \beta_{\alpha}(Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n},P_{\mathbf{Z},\textnormal{PPM}}^{n,\ell})\geqslant{\mathbb{P}}{\left(\log\frac{P_{\mathbf{Z},\textnormal{PPM}}^{n,\ell}(\widetilde{\mathbf{Z}})}{Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}(\widetilde{\mathbf{Z}})}\leqslant\gamma\right)} where $\mathbf{Z}$ and $\widetilde{\mathbf{Z}}$ are defined as before. Using Theorem 6 again, similar to (162), we obtain

[TABLE]

Accordingly, to ensure that {\mathbb{P}}{\left(\log\frac{P_{\mathbf{Z},\textnormal{PPM}}^{n,\ell}({\mathbf{Z}})}{Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}({\mathbf{Z}})}\leqslant\gamma\right)}\geqslant\alpha, we choose $\gamma=\sqrt{\frac{\ell\chi_{2}(Q_{1}\|Q_{0})}{m}}Q^{-1}{\left(\alpha+\frac{1}{\sqrt{\ell}}\right)}-\frac{\ell\chi_{2}(Q_{1}\|Q_{0})}{2m}+O{\left(\frac{1}{\sqrt{n}}\right)}$. Thus, we have

[TABLE]

∎

III-D Converse

We now develop a generic converse for a quasi-metric $d$. The following result states that the number of covert and reliable bits that one can transmit may be characterized by establishing an upper bound on the weight of codewords. Such upper bounds are metric-specific, and we develop them in Section IV.

Theorem 7**.**

Let $d$ be a quasi-metric and $({\mathcal{X}},W_{Y|X},W_{Z|X},{\mathcal{Y}},{\mathcal{Z}})$ be a DMC. If there exists a function $g:\mathbb{R}\to\mathbb{R}$ such that for every $(M,K,n,\epsilon,\delta)_{d}$ code $\mathcal{C}$, there exists a subset of the codewords, $\mathcal{D}$, with

[TABLE]

and $|{\mathcal{D}}|\geqslant\frac{MK}{n^{h}}$ where $C,h>0$ only depend on the channel, then

[TABLE]

In addition, if for every $(M_{n},K_{n},n,\epsilon,\delta)_{d}$ code in a sequence $\{{\mathcal{C}}_{n}\}$ of increasing length $n$ such that $\lim_{n\rightarrow\infty}\epsilon_{n}=0$, there exists a non-empty subset of codewords $\mathcal{D}_{n}\subset{\mathcal{C}}_{n}$ with

[TABLE]

Then,

[TABLE]

Proof:

Let $\mathcal{C}$ be an $(M,K,n,\epsilon,\delta)_{d}$ code. For $s\in\llbracket{1},{K}\rrbracket$, we denote the sub-codebook of all codewords characterized by the key value $s$ by $\mathcal{C}^{s}$. By our assumptions, we can choose $\mathcal{D}\subset\mathcal{C}$ of size at least $MK/n^{h}$ satisfying (167). By the pigeonhole principle, there should be at least one sub-codebook $\mathcal{C}^{s}$ such that $|\mathcal{D}\cap\mathcal{C}^{s}|\geqslant M/n^{h}$. Let us define

[TABLE]

Since $\mathcal{D}^{s}_{i}$ is included in $\mathcal{C}^{s}$, it is a reliability code for the channel $W_{Y|X}$ with maximum probability of error less than or equal to $\epsilon$. Moreover, the type of the codewords in $\mathcal{D}^{s}_{i}$ is fixed, and therefore for any $\textbf{x}\in\mathcal{D}_{i}^{s}$, [18, Theorem 28] yields that

[TABLE]

Furthermore, similar to [19], by [18, Equation 102], we obtain for any $\gamma>0$

[TABLE]

However, we know that $W_{\mathbf{Y}|\mathbf{X}}(\mathbf{Y}|\textbf{x})=\prod_{k:x_{k}=1}P_{1}(Y_{k})\prod_{j:x_{j}=0}P_{0}(Y_{j})$ which yields that if $\tilde{Y}_{1},\cdots,\tilde{Y}_{i}$ are i.i.d. according to $P_{1}$, then

[TABLE]

By Theorem 6, we have

[TABLE]

where $B$ just depends on the channel. Setting $\gamma=iD_{P}+\sqrt{iV_{P}}Q^{-1}(1-\epsilon-\frac{2B}{\sqrt{i}})$ and combining all above equations, we have

[TABLE]

Hence, we obtain

[TABLE]

Thus, we get

[TABLE]

Note that we need not worry about exotic channels for $\epsilon\in(\frac{1}{2},1)$ since the constants $D_{P}$ and $V_{P}$ are known and fixed.

Consider now a sequence of $(M_{n},K_{n},n,\epsilon_{n},\delta)$ code for which subsets ${\mathcal{D}}_{n}\subset{\mathcal{C}}_{n}$ exist and (169) holds. Recall that $\epsilon_{n}$ is defined as per (16). By the pigeon hole principle, for every $n$, there exists a sub-codebook ${\mathcal{C}}_{n}(s)$ consisting of all codewords indexed by the same key $s\in\llbracket{1},{K_{n}}\rrbracket$, such that $\left|{{\mathcal{C}}_{n}(s)\cap{\mathcal{D}}_{n}}\right|\geqslant\frac{\left|{{\mathcal{D}}_{n}}\right|}{K_{n}}$, and the maximum probability of error corresponding to ${\mathcal{C}}_{n}(s)\cap{\mathcal{D}}_{n}$ is at most $\epsilon_{n}$. Following the exact same converse steps as in [10, Section VI], we obtain for $n$ large enough that

[TABLE]

Consequently,

[TABLE]

and

[TABLE]

∎

IV Covert Communication with specific covertness metrics

We now leverage the general results established in Section III and specialize them to study three covertness metrics: relative entropy (Subsection IV-A), variational distance (Subsection IV-B), and probability of missed detection (Subsection IV-C). As alluded to earlier, all that needs to be done is: (i) establish that the metric under consideration is a quasi metric, as defined at the beginning of Section III; (ii) verify that the conditions of Theorem 5 and Theorem 7 are satisfied.

IV-A Covertness in relative entropy

In this subsection, we prove Theorem 1.

Lemma 9**.**

Relative entropy is a quasi-metric, i.e.,

[TABLE]

Proof:

Note that

[TABLE]

∎

Proof:

Fix $\epsilon\in]0,1[$ and $\delta>0$, and define $\omega\triangleq\sqrt{\frac{2\delta}{\chi_{2}(Q_{1}\|Q_{0})}}$ and $\ell_{n}\triangleq\lfloor\omega\sqrt{n}-t\rfloor$ where the value of $t$ will be determined later. To use Theorem 5, we choose $P_{\mathbf{X}}^{n}\triangleq P_{\mathbf{X},\textnormal{PPM}}^{n,\ell_{n}}$, and we set

[TABLE]

and

[TABLE]

By Lemma 8, we know that for $n$ large enough and some $C>0$

[TABLE]

where $(a)$ is true for $t>\frac{1+\omega^{3}C}{\omega\chi_{2}(Q_{1}\|Q_{0})}$ and $n$ large enough. Furthermore, by Lemma 6, we have

[TABLE]

Hence, for large enough $n$, we have

[TABLE]

where $(a)$ follows from Lemma 5. By Lemma 7, for any $\rho>0$, we have

[TABLE]

where $(a)$ is true for large enough $n$. Thus, all the conditions in Theorem 5 hold, and we obtain

[TABLE]

∎

Proof:

We only focus on the converse for second-order asymptotics since the first-order result follows from [10, 11]. For an $(M,K,n,\epsilon,\delta)_{D}$ code, we briefly outline for completeness the argument of [10] to obtain an upper-bound on the average weight of the codewords. Note that, by [10, Equation (96)],

[TABLE]

where $\overline{P}_{Z}(z)\triangleq\frac{1}{n}\widehat{P}_{Z_{i}}(z)$. We know that $\overline{P}_{Z}=\mu Q_{1}+(1-\mu)Q_{0}$ for $\mu\triangleq\frac{1}{MK}\sum_{w\in\llbracket{1},{M}\rrbracket,s\in\llbracket{1},{K}\rrbracket}\frac{\textnormal{wt}(\mathbf{x}_{s,w})}{n}$. Therefore, by [10, Equation (11)], we have

[TABLE]

Combining these two bounds, we obtain

[TABLE]

for some $B>0$ depending on the channel. Thus, we can choose a subset ${\mathcal{D}}$ of codewords of size $MK/n$ such that

[TABLE]

Applying Theorem 7 completes the converse proof. ∎

IV-B Covertness with variational distance

In this subsection, we prove Theorem 2.

Lemma 10**.**

Total variation is a quasi-metric for distributions, i.e., for all distributions $P$, $Q$, and $R$ defined over ${\mathcal{Z}}$, we have

[TABLE]

Proof:

Total variation is a metric, and therefore we have

[TABLE]

where $(a)$ follows from Pinsker’s Inequality. ∎

Proof:

Fix $\epsilon,\delta\in]0,1[$ and $n$. Similar to the analysis for relative entropy, we use Theorem 5 with $P_{\mathbf{X}}^{n}\triangleq P_{\mathbf{X},\textnormal{PPM}}^{n,\ell_{n}}$ where $\ell_{n}$ is chosen to ensure {{\mathbb{V}}\!\left(P_{\mathbf{Z},\textnormal{PPM}}^{n,\ell_{n}},Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}\right)}\leqslant\delta-\frac{1}{\sqrt{n}}. Specifically, upon setting $\Gamma\triangleq Q^{-1}{\left(\frac{1-\delta}{2}\right)}$, we wish to choose $\ell_{n}$ such that

[TABLE]

The term $\Gamma$ is what we would expect by inspection of (136) to obtain a variational distance with first order approximately $\delta$. As we shall see next, the penalty term $\frac{e^{\frac{\Gamma^{2}}{2}}\sqrt{2\pi}}{\sqrt{\ell_{n}}}$ is the result of a Taylor series of the $Q$ function around $\Gamma$. Formally, we choose $\sqrt{\ell_{n}}$ as a solution of the cubic equation

[TABLE]

which will be sufficient to ensure (217). For a cubic equation of the form $x^{3}-px+q=0$, the roots are known in closed algebraic form and one of them is $2\sqrt{\frac{-p}{3}}\cos{\left(\frac{1}{3}\arccos{\left(\frac{3q}{2p}\sqrt{\frac{-3}{p}}\right)}\right)}$. Thus,

[TABLE]

where $(a)$ and $(b)$ follow since, for $x$ close to zero, $\arccos(x)=\frac{\pi}{2}-x+O(x^{2})$ and $\cos{\left(\frac{\pi}{6}+x\right)}=\frac{\sqrt{3}}{2}-\frac{1}{2}x+O(x^{2})$, respectively. Consequently, upon choosing

[TABLE]

we satisfy (217). Combining with (136), we have

[TABLE]

where $(a)$ follows from $Q(x-y)\geqslant Q(x)+\frac{e^{-\frac{x^{2}}{2}}y}{\sqrt{2\pi}}$ for $x\geqslant y\geqslant 0$. Finally, we can control the term $O{\left(\frac{1}{\sqrt{n}}\right)}$ in (217) to guarantee that (227) is less than $\delta-\frac{1}{\sqrt{n}}$.

Finally, let

[TABLE]

and for some $\rho>0$,

[TABLE]

Similar to (203), we can show that, for large enough $n$,

[TABLE]

and

[TABLE]

Thus, all the conditions in Theorem 5 hold, and we obtain

[TABLE]

∎

To develop the converse for variational distance, we start by relating the variational distance {{\mathbb{V}}\!\left(\widehat{P}_{\mathbf{Z}},Q_{0}^{{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}}\right)} to the minimum weight of the codewords.

Lemma 11**.**

Consider a binary-input DMC $({\mathcal{X}},W_{Z|X},{\mathcal{Z}})$ and $M$ codewords $\textbf{x}_{1},\cdots,\textbf{x}_{M}\in\mathcal{X}^{n}$ with induced distribution $\widehat{P}_{\mathbf{Z}}$ on $\mathcal{Z}^{n}$. If $w_{\min}\triangleq\min_{m\in\llbracket{1},{M}\rrbracket}\textnormal{wt}(\textbf{x}_{m})$, then we have

[TABLE]

where $B$ is a constant that only depends on the channel.

Proof:

To lower bound {{\mathbb{V}}\!\left(\widehat{P}_{\mathbf{Z}},Q_{0}^{{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}}\right)}, we introduce a hypothesis testing problem with two hypotheses $H_{0}$ and $H_{1}$ corresponding to distributions Q_{0}^{{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}} and $\widehat{P}_{\mathbf{Z}}$, respectively. We know that for any test with probability of false alarm and missed detection $\alpha$ and $\beta$, respectively, we have

[TABLE]

Hence, to link the variational distance to the weight of codewords, it suffices to introduce a test for which $\alpha$ and $\beta$ conveniently relate to the weight of codewords. We consider here the sub-optimal test

[TABLE]

where $A(z)\triangleq\frac{Q_{1}(z)-Q_{0}(z)}{Q_{0}(z)}$ and $\tau$ is an arbitrary constant that is determined later. Intuitively, this test plays the same role for DMCs as the radiometer played for Gaussian channels [6] in that it only depends on the codeword weight. To bound the probability of false alarm, we use Theorem 6 to obtain

[TABLE]

with

[TABLE]

Note that all above quantities are finite. For the probability of missed detection, we condition on the codeword transmitted by the channel to obtain with Theorem 6

[TABLE]

with

[TABLE]

If we choose $\tau=n\mu_{0}+\frac{w_{\min}}{2}(\mu_{1}-\mu_{0})$, we have

[TABLE]

Note that if we have $\sigma_{1}\leqslant\sigma_{0}$, then we get

[TABLE]

Otherwise $\sigma_{1}>\sigma_{0}$ and we split the summation in (243) into two parts: if $\textnormal{wt}(\textbf{x}_{m})>\frac{\sigma_{1}}{\sigma_{0}}w_{\min}$, then we have

[TABLE]

where $(a)$ follows from $\sigma_{1}>\sigma_{0}$. If $w_{\min}\leqslant\textnormal{wt}(\textbf{x}_{m})\leqslant\frac{\sigma_{1}}{\sigma_{0}}w_{\min}$, then we obtain

[TABLE]

for $B_{2}=\frac{\sigma_{1}(\sigma_{1}^{2}-\sigma_{0}^{2})}{2\sigma_{0}^{3}}>0$ and $B_{3}=\frac{(\mu_{1}-\mu_{0})B_{2}}{2\sqrt{2\pi}\sigma_{0}}>0$ where $(a)$ follows from $\frac{1}{\sqrt{1+x}}\geqslant-\frac{x}{2}+1$ for all $x>0$, and $(b)$ follow from $Q(x-y)\leqslant Q(x)+\frac{y}{\sqrt{2\pi}}$ for all $0<y<x$. Therefore, we always have

[TABLE]

Moreover, plugging in the value $\tau$ in (237), we obtain

[TABLE]

for some positive constant $B_{0}$. Using (235), (251), and (252), we obtain

[TABLE]

Finally, since $\mu_{0}=0$ and $\sigma_{0}^{2}=\mu_{1}=\chi_{2}(Q_{1}\|Q_{0})$, we have

[TABLE]

∎

Next, we develop a bound for the maximum weight of a sub-codebook.

Lemma 12**.**

Let $\mathcal{C}$ be an $(M,K,n,\epsilon,\delta)_{V}$ code for a binary-input covert communication channel $({\mathcal{X}},W_{Y|X},W_{Z|X},{\mathcal{Y}},{\mathcal{Z}})$. For all $\gamma\in[0,1]$, there exists a subset of codewords $\mathcal{D}$ such that $|\mathcal{D}|\geqslant\gamma MK$ and

[TABLE]

where $C$ is a constant that depends only on the channel.

Proof:

We define

[TABLE]

where $C_{2}>0$ is specified later and $\mathcal{D}\triangleq\{\textbf{x}\in\mathcal{C}:\textnormal{wt}(\textbf{x})\leqslant A\sqrt{n}\}$. The set $\mathcal{D}$ satisfies (255), and we just need to check $|\mathcal{D}|\geqslant\gamma MK$. To this end, let $\widehat{P}_{1}$ and $\widehat{P}_{2}$ be the induced output distributions for codes $\mathcal{D}$ and $\mathcal{C}\setminus\mathcal{D}$, respectively. Then, we have

[TABLE]

where $(a)$ follows from the definition of an $(M,K,n,\epsilon,\delta)_{V}$ code, $(b)$ follows from the triangle inequality and $\widehat{P}_{\mathbf{Z}}=\frac{|\mathcal{D}|}{MK}\widehat{P}_{1}+(1-\frac{|\mathcal{D}|}{MK})\widehat{P}_{2}$, $(c)$ follows from Lemma 11, $(d)$ follows from the definition of $A$, and $(e)$ follows by choosing

[TABLE]

Therefore, we get $|\mathcal{D}|\geqslant\gamma MK$. ∎

We can now establish the converse.

Proof:

In Lemma 12, if we set $\gamma=\frac{1}{\sqrt{n}}$, for any $(M,K,n,\epsilon,\delta)_{V}$ code, we obtain a subset of codewords $\mathcal{D}$ with $|\mathcal{D}|\geqslant MK/\sqrt{n}$ and

[TABLE]

Therefore, by Theorem 7, we have

[TABLE]

Lemma 12 already ensures the upper bound needed to apply Theorem 7, and we only need to make sure that we can choose $\gamma$ in Lemma 12 such that we can satisfy the other conditions in (169). For every $(M_{n},K_{n},n,\epsilon_{n},\delta)_{V}$ code in a sequence of codes such that $\lim_{n\to\infty}\epsilon_{n}=0$, we choose $\gamma_{n}=\frac{1}{n}$. Then,

[TABLE]

Hence, the results follows by Theorem 7.

∎

IV-C Covertness with probability of missed detection at fixed significance level

In this subsection, we prove Theorem 3.

Lemma 13**.**

For a fixed $\alpha\in]0,1[$ and two distributions $P$ and $Q$ over same set $\mathcal{Z}$, we define $d(P,Q)\triangleq 1-\alpha-\beta_{\alpha}(Q,P)$; then $d$ is a quasi-metric, and we have

[TABLE]

Proof:

By definition of $\beta_{\alpha}(Q,R)$, there exists a set $\mathcal{T}$ with $Q(\mathcal{T})\leqslant\alpha$ and $R(\mathcal{T})=\beta_{\alpha}(Q,R)$; therefore we have

[TABLE]

By Pinsker’s Inequality, we obtain

[TABLE]

Finally, using the definition of $d(P,Q)$, we get the result. ∎

Proof:

Fix $\epsilon\in]0,1[$, $\delta\in]0,1-\alpha[$, and $n$; we just show how we can choose $\ell_{n}$ such that \beta_{\alpha}(Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n},P_{\mathbf{Z},\textnormal{PPM}}^{n,\ell_{n}})\geqslant 1-\alpha-\delta+\frac{1}{\sqrt{n}} and the rest of the proof is similar to the proof for variational distance. Let $\Lambda\triangleq Q^{-1}(1-\alpha-\delta)$ and $\Upsilon\triangleq Q^{-1}(\alpha)$; we wish to choose $\ell_{n}$ to satisfy

[TABLE]

Using $Q^{-1}{\left(\alpha+x\right)}=Q^{-1}(\alpha)-\sqrt{2\pi}e^{\frac{{\left(Q^{-1}(\alpha)\right)}^{2}}{2}}x+O(x^{2})=\Upsilon+\sqrt{2\pi}e^{\frac{\Upsilon^{2}}{2}}x+O(x^{2})$ for $x$ close to zero, the above inequality is equivalent to

[TABLE]

We form the cubic equation

[TABLE]

and set $\sqrt{\ell_{n}}$ to be its solution; Thus, similar to (222), we obtain

[TABLE]

Hence, if we choose

[TABLE]

we have

[TABLE]

where $(a)$ follows from (137), $(b)$ follows from (274), and $(c)$ follows from $Q(\Lambda+x)=Q(\Lambda)+\frac{x}{\sqrt{2\pi}e^{\frac{\Lambda^{2}}{2}}}+O(x^{2})$ for $x$ close to zero. ∎

The proof of the converse of Theorem 3 requires the following steps similar to those of covertness with variational distance. We first relate the probability of missed detection \beta_{\alpha}\left(Q_{0}^{{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}},\widehat{P}_{\mathbf{Z}}\right) at significance level $\alpha$ to the minimum weight of codewords.

Lemma 14**.**

Consider a binary-input DMC $({\mathcal{X}},W_{Z|X},{\mathcal{Z}})$ and $M$ codewords $\textbf{x}_{1},\cdots,\textbf{x}_{M}\in\mathcal{X}^{n}$ with induced distribution $\widehat{P}_{\mathbf{Z}}$ on $\mathcal{Z}^{n}$. If $\frac{\sqrt{n}Q^{-1}(\alpha)}{\sqrt{\chi_{2}(Q_{1}\|Q_{0})}}<w_{\min}\triangleq\min_{m\in\llbracket{1},{M}\rrbracket}\textnormal{wt}(\textbf{x}_{m})$, we have

[TABLE]

where $B$ is a constant that just depends on the channel.

Proof:

For the hypothesis testing problem consisting of two hypotheses $H_{0}$ with distribution Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n} and $H_{1}$ with distribution $\widehat{P}_{\mathbf{Z}}$, we introduce again the test

[TABLE]

where $A(z)\triangleq\frac{Q_{1}(z)-Q_{0}(z)}{Q_{0}(z)}$. Using Theorem 6 and the calculations in the proof of Lemma 11, we have

[TABLE]

Thus, if we choose $\tau=\sqrt{n\chi_{2}(Q_{1}\|Q_{0})}Q^{-1}\left(\alpha+O\left(n^{-\frac{1}{2}}\right)\right)$, the false alarm probability would be less than or equal to $\alpha$. Hence, by definition of \beta_{\alpha}\left(Q_{0}^{{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}},\widehat{P}_{\mathbf{Z}}\right), we get

[TABLE]

with

[TABLE]

Plugging in the value of $\tau$, we obtain

[TABLE]

Analogous to the proof of Lemma 11, we obtain

[TABLE]

∎

Next, we develop an upper bound on the weight of codewords as a function of channel characteristics and covertness measured in terms of probability of missed detection.

Lemma 15**.**

Let $({\mathcal{X}},W_{Y|X},W_{Z|X},{\mathcal{Y}},{\mathcal{Z}})$ be a binary-input covert communication channel and $\mathcal{C}$ be an $(M,K,n,\epsilon,\delta,\alpha)_{\beta}$ code. For all $\gamma\in[0,1]$, we can choose a subset of codewords $\mathcal{D}$ such that $|\mathcal{D}|\geqslant\gamma MK$ and

[TABLE]

where $C$ is a constant that depends just on the channel.

Proof:

If $A\triangleq\frac{1}{\sqrt{\chi_{2}(Q_{1}\|Q_{0})}}\left(Q^{-1}\left(1-\alpha-\delta-\frac{C_{2}}{\sqrt{n}}-\gamma\right)+Q^{-1}(\alpha)\right)$ for $C_{2}>0$ specified later, we define

[TABLE]

Obviously, $\mathcal{D}$ satisfies (296), and we just need to check $|\mathcal{D}|\geqslant\gamma MK$. To do so, let $\widehat{P}_{1}$ and $\widehat{P}_{2}$ be the induced output distributions for codes $\mathcal{D}$ and $\mathcal{C}\setminus\mathcal{D}$, respectively. Using Lemma 14 for $\mathcal{C}\setminus\mathcal{D}$, we get

[TABLE]

where $(a)$ follows from the definition of $A$ and by choosing

[TABLE]

This means that there exists $\mathcal{T}\subset\mathcal{Z}^{n}$ such that Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}n}(\mathcal{T})\leqslant\alpha and $\widehat{P}_{2}(\mathcal{T})\leqslant 1-\alpha-\delta-\gamma$. Accordingly, we have

[TABLE]

where $(a)$ follows from the definition of an $(M,K,n,\epsilon,\delta,\alpha)_{\beta}$ code, and $(b)$ follows from $\widehat{P}_{\mathbf{Z}}=\frac{|\mathcal{D}|}{MK}\widehat{P}_{1}+(1-\frac{|\mathcal{D}|}{MK})\widehat{P}_{2}$. Simplifying the above inequality, we get $|\mathcal{D}|\geqslant\gamma MK$. ∎

The converse then proceeds as for the converse of Theorem 2 in the previous section.

V Conclusion

We have developed an approach to study covert communication when covertness is measured with several “quasi-metrics,” as defined in Section III. It is legitimate to ask which metric would make most sense from an operational perspective. While relative entropy is amenable to a fairly extensive information-theoretic analysis, as illustrated by our complete characterization of second-order asymptotics in Theorem 1, variational distance and probability of false alarm are probably more adequate since they directly relate to the operation of an adversary attempting to detect communication. Since measuring covertness in terms of variational distance does not impose any constraint on where the adversary operates on its ROC curve, we believe that variational distance is perhaps the most operationally relevant covertness metric.

Our results have been restricted to binary-input DMCs to obtain simple closed-form expressions but extensions to arbitrary finite input alphabets may be obtained following the approach of [11]; however several research questions remain open. We have not characterized the exact second-order asymptotics of covert communication with variational distance and probability of missed detection metrics; we conjecture, however, that the upper-bounds are achievable. We have also not characterized the second-order asymptotics for the number of key bits required; a close inspection of our current proof technique shows that we explicitly rely on a law of large numbers to analyze the number of key bits, which one would have to circumvent.

Appendix A The Effect of Average Probability of Error on First Order Asymptotics

Assume that the optimal second order asymptotics are of the form

[TABLE]

for some function $f$ that is strictly increasing in $\delta$ and does not depend on $\epsilon$, which is what we would hope to establish based on our results maximum probability of error. Now for $1>\epsilon>0$ and $\delta>0$, pick $\frac{\epsilon}{1-\epsilon}>\alpha>0$ and set $\epsilon^{\prime}=(1+\alpha)\epsilon-\alpha$, and $\delta^{\prime}=(1+\alpha)\delta$. Since $\delta^{\prime}>\delta$ and because of (305), we expect for all blocklengths large enough that

[TABLE]

Consider now a code with $M^{*}(\epsilon^{\prime},\delta^{\prime})$ codewords, and let $\widehat{Q^{\prime}}^{n}$ be the distribution induced at the eavesdropper’s output when randomizing uniformly over all codewords. Let $\mathcal{S}$ be a set (with repetition) of $N\triangleq\alpha M^{*}(\epsilon^{\prime},\delta^{\prime})$ all-zero codewords. We construct a new code by adding the set $\mathcal{S}$ to the previous code. The resulting code has average probability of error

[TABLE]

It also induces a new distribution $\widehat{Q}^{n}$ at the eavesdropper’s output when randomizing uniformly over all codewords, such that

[TABLE]

We therefore obtain a new code with probability of error less than $\epsilon$ and relative entropy less than $\delta$, but with a number of codewords M^{*}(\epsilon^{\prime},\delta^{\prime})+{\color[rgb]{0,0,0}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,0}\pgfsys@color@gray@stroke{0}\pgfsys@color@gray@fill{0}N} that exceeds $M^{*}(\epsilon,\delta)$. This contradiction shows that (305) does not hold, so that either $f(\delta)$ is non-increasing or $\delta$ or depends on $\epsilon$. In both cases, this suggests a result significantly more complex that what we were trying to establish. Note that this does not contradict existing results on covert capacity [11, 10] because these works have all been in the regime $\lim_{n\rightarrow\infty}\epsilon=0$. Finally, from this example, we conclude that no strong converse exists when measuring reliability with an average probability of error and even the first-order asymptotics depend on a non-vanishing average probability of error. Hence, one could expect a much more challenging analysis of second-order asymptotics under average probability of error constraint.

Appendix B A technical lemma

Lemma 16**.**

Suppose $\mathbf{Z}\triangleq(Z_{1},\cdots,Z_{m})$ is distributed according to Q_{0}^{\mathchoice{\raisebox{1.0pt}{\displaystyle\otimes}}{\raisebox{1.0pt}{\otimes}}{\raisebox{0.5pt}{\scalebox{0.7}{\scriptstyle\otimes}}}{\raisebox{0.4pt}{\scalebox{0.6}{\scriptscriptstyle\otimes}}}m}. Define $A(z)\triangleq\frac{Q_{1}(z)-Q_{0}(z)}{Q_{0}(z)}$ and let $B\triangleq\frac{1}{m}\sum_{i=1}^{m}A(Z_{i})$. Also define $\underline{B}$ with distribution ${\mathbb{P}}{\left(\underline{B}=b\right)}\triangleq{\mathbb{P}}{\left(B=b|B\neq-1\right)}$ and $\underline{C}\triangleq\log(1+\underline{B})$. There exists a constant $0\leqslant\tau<1$ depending only on the channel such that

[TABLE]

and

[TABLE]

Suppose $\widetilde{\mathbf{Z}}\triangleq(\widetilde{Z}_{1},\cdots,\widetilde{Z}_{m})$ is distributed according to $P_{\mathbf{Z},\textnormal{PPM}}^{m,1}$. Define $\widetilde{B}\triangleq\frac{1}{m}\sum_{i=1}^{m}A(\widetilde{Z}_{i})$, $\underline{\widetilde{B}}$ with distribution ${\mathbb{P}}{\left(\widetilde{\underline{B}}=b\right)}\triangleq{\mathbb{P}}{\left(\widetilde{B}=b|\widetilde{B}\neq-1\right)}$ and $\widetilde{\underline{C}}\triangleq\log(1+\underline{\widetilde{B}})$. Then

[TABLE]

and

[TABLE]

Proof:

We first show that the moments of $B$ and $\underline{B}$ are close. Define $\kappa\triangleq\max_{z:Q_{0}(z)>0}\left|\frac{Q_{1}(z)-Q_{0}(z)}{Q_{0}(z)}\right|$ and $\tau\triangleq{\mathbb{P}_{Q_{0}}}{\left(A(Z)=-1\right)}$. Notice that ${{\mathbb{V}}\!\left(P_{B},P_{\underline{B}}\right)}=\tau^{m}$, $\left|B\right|\leqslant\kappa$ and $\left|\underline{B}\right|\leqslant\kappa$ almost surely and that $\kappa$ does not depend on $m$. For any $q\geqslant 1$, we therefore obtain

[TABLE]

Therefore, it is sufficient to find ${\mathbb{E}}{\left(B^{i}\right)}$ for $i=1,2,3,4$. Note that

[TABLE]

Hence, we obtain

[TABLE]

To find the expected value of $\underline{C}$, note that $\ln(1+x)\leqslant x-\frac{x^{2}}{2}+\frac{x^{3}}{3}$, and therefore, we have

[TABLE]

For any $0<a<1$ and $x>-a$, we also have $x-\frac{x^{2}}{2}+\frac{x^{3}}{3}-\frac{x^{4}}{4(1-a)^{4}}\leqslant\ln(1+x)$; thus,

[TABLE]

where $(a)$ follows since $\min_{z:Q_{0}(z)>0,Q_{1}(z)>0}A(z)>-1$. Furthermore, for the variance of $\underline{C}$, since $\log^{2}(1+x)\geqslant{\left(x-\frac{x^{2}}{2}\right)}^{2}-x^{4}$, we have

[TABLE]

Moreover, for $-1<a<0$ and $x>-a$, we have $\log^{2}(1+x)\leqslant{\left(x-\frac{x^{2}}{2}+\frac{x^{3}}{3(1-a)^{3}}\right)}^{2}$. Therefore,

[TABLE]

Finally, for the third moment of $C$, note that

[TABLE]

where $(a)$ follows from Jensen’s inequality.

We now calculate the moments with respect to PPM distribution. First note that if we define $P_{\mathbf{Z}}^{i}(\mathbf{z})\triangleq Q_{1}(z_{i})\prod_{j\neq i}Q_{0}(z_{j})$, then we can write $P_{\mathbf{Z},\textnormal{PPM}}^{m,1}=\frac{1}{m}P_{\mathbf{Z}}^{i}$; therefore, for any function of $\widetilde{\mathbf{Z}}$ such as $f(\widetilde{\mathbf{Z}})$, we have ${\mathbb{E}_{P_{\mathbf{Z},\textnormal{PPM}}^{m,1}}}{\left(f(\widetilde{\mathbf{Z}})\right)}=\frac{1}{m}\sum_{i=1}^{m}{\mathbb{E}_{P_{\mathbf{Z}}^{i}}}{\left(f(\widetilde{\mathbf{Z}})\right)}$. Moreover, if $f$ is invariant with respect to all permutation of elements of $\mathbf{z}$, then ${\mathbb{E}_{P_{\mathbf{Z},\textnormal{PPM}}^{m,1}}}{\left(f(\widetilde{\mathbf{Z}})\right)}={\mathbb{E}_{P_{\mathbf{Z}}^{1}}}{\left(f(\widetilde{\mathbf{Z}})\right)}$. One can check that this property holds for all moments of $\widetilde{B}$ and $\widetilde{C}$. Therefore, in the sequel, we assume that $\widetilde{\mathbf{Z}}$ is distributed according to $P_{\mathbf{Z}}^{1}$. Hence, we have

[TABLE]

One can show that $\left|{\mathbb{E}}{\left(\smash{\widetilde{\underline{B}}}^{q}\right)}-{\mathbb{E}}{\left(\widetilde{B}^{q}\right)}\right|=O(\tau^{m})$ as was done earlier for $B$ and $\underline{B}$. Using then the same bounds for $\log(1+x)$ as before and the above calculations, we obtain the desired bounds for the moments of $\widetilde{C}$. ∎