A Unified Framework for One-shot Achievability via the Poisson Matching Lemma

TL;DR

This paper introduces the Poisson matching lemma, a new fundamental tool that simplifies and improves one-shot achievability bounds across various information theory problems, extending previous work on Poisson functional representation.

Contribution

The paper presents the Poisson matching lemma and demonstrates its broad applicability, providing improved one-shot bounds and simpler proofs in network information theory.

Findings

Improved one-shot bounds in most settings compared to previous results.

Simplified proofs by replacing packing and covering lemmas.

Extension of Poisson functional representation to fixed-length settings.

Abstract

We introduce a fundamental lemma called the Poisson matching lemma, and apply it to prove one-shot achievability results for various settings, namely channels with state information at the encoder, lossy source coding with side information at the decoder, joint source-channel coding, broadcast channels, distributed lossy source coding, multiple access channels, channel resolvability and wiretap channels. Our one-shot bounds improve upon the best known one-shot bounds in most of the aforementioned settings (except multiple access channels, channel resolvability and wiretap channels, where we recover bounds comparable to the best known bounds), with shorter proofs in some settings even when compared to the conventional asymptotic approach using typicality. The Poisson matching lemma replaces both the packing and covering lemmas, greatly simplifying the error analysis. This paper extends the work of Li and El Gamal on Poisson functional representation, which mainly considered variable-length source coding settings, whereas this paper studies fixed-length settings, and is not limited to source coding, showing that the Poisson functional representation is a viable alternative to typicality for most problems in network information theory.

Equations

$${\iota }_{X;Y|Z}(x;y|z):=\log \frac{d{P}_{XY|Z=z}}{d(P_{X|Z=z}\times P_{Y|Z=z})}(x,y).$$

$$\frac{d\nu }{d\mu }:\mathcal{X}\to [0,\infty ).$$

$$\frac{d{\nu }_1}{d{\nu }_2}(x)=\frac{d{\nu }_1}{d\mu }(x){\left(\frac{d{\nu }_2}{d\mu }(x)\right)}^{-1}\in [0,\infty ],$$

$${\tilde{U}}_P\left(\{{\bar{U}}_i,T_i{\}}_{i\in \mathbb{N}}\right):={\bar{U}}_{K_P(\{{\bar{U}}_i,T_i{\}}_{i\in \mathbb{N}})},$$

$$K_P\left(\{{\bar{U}}_i,T_i{\}}_{i\in \mathbb{N}}\right):=\underset{i:\frac{dP}{d\mu }({\bar{U}}_i)>0}{\mathrm{argmin}}{T}_i{\left(\frac{dP}{d\mu }({\bar{U}}_i)\right)}^{-1},$$

$$\mathbf{P}\left\{{\tilde{U}}_Q\ne {\tilde{U}}_P|{\tilde{U}}_P\right\}\le 1-{\left(1+\frac{dP}{dQ}({\tilde{U}}_P)\right)}^{-1},$$

$$\mathbf{P}\left\{{\tilde{U}}_{Q_{U|Y}(\cdot |Y)}\ne U|X,U,Y\right\}\le 1-{\left(1+\frac{d{P}_{U|X}(\cdot |X)}{d{Q}_{U|Y}(\cdot |Y)}(U)\right)}^{-1}.$$

$$P_e\le \mathbf{E}\left[1-{\left(1+\mathsf{L}{2}^{-{\iota }_{X;Y}(X;Y)}\right)}^{-1}\right]$$

$$\mathbf{P}\left\{M\ne {\tilde{M}}_{P_{X|Y}(\cdot |Y)\times P_M}\right\}$$

$$\le \mathbf{P}\left\{(X,M)\ne (\tilde{X},\tilde{M}{)}_{P_{X|Y}(\cdot |Y)\times P_M}\right\}$$

$$=\mathbf{E}\left[\mathbf{P}\left\{(X,M)\ne (\tilde{X},\tilde{M}{)}_{P_{X|Y}(\cdot |Y)\times P_M}|M,X,Y\right\}\right]$$

$$\stackrel{(a)}{\le }\mathbf{E}\left[1-{\left(1+\frac{d{P}_X\times {\delta }_M}{d{P}_{X|Y}(\cdot |Y)\times P_M}(X,M)\right)}^{-1}\right]$$

$$=\mathbf{E}\left[1-(1+\mathsf{L}{2}^{-{\iota }_{X;Y}(X;Y)}{)}^{-1}\right],$$

$$P_e\le \mathbf{E}\left[1-{\left(1-\min \left\{2^{-{\iota }_{X;Y}(X;Y)},1\right\}\right)}^{(\mathsf{L}+1)/2}\right]$$

$$P_e\le \mathbf{E}\left[\min \left\{\frac{\mathsf{L}-1}{2}\cdot 2^{-{\iota }_{X;Y}(X;Y)},1\right\}\right].$$

$$\mathbf{E}\left[1-{\left(1-\min \left\{2^{-{\iota }_{X;Y}(X;Y)},1\right\}\right)}^{(\mathsf{L}+1)/2}\right]$$

$$\le \mathbf{E}\left[\min \left\{\frac{\mathsf{L}+1}{2}\cdot 2^{-{\iota }_{X;Y}(X;Y)},1\right\}\right].$$

$$P_e\le \mathbf{E}\left[1-(1+\mathsf{L}{2}^{{\iota }_{U;S}(U;S)-{\iota }_{U;Y}(U;Y)}{)}^{-1}\right]$$

$$\mathbf{P}\{M\ne {\tilde{M}}_{P_{U|Y}(\cdot |Y)\times P_M}\}$$

$$\le \mathbf{P}\{(U,M)\ne (\tilde{U},\tilde{M}{)}_{P_{U|Y}(\cdot |Y)\times P_M}\}$$

$$=\mathbf{E}\left[\mathbf{P}\left\{(U,M)\ne (\tilde{U},\tilde{M}{)}_{P_{U|Y}(\cdot |Y)\times P_M}|M,S,U,Y\right\}\right]$$

$$\stackrel{(a)}{\le }\mathbf{E}\left[1-{\left(1+\frac{d{P}_{U|S}(\cdot |S)\times {\delta }_M}{d{P}_{U|Y}(\cdot |Y)\times P_M}(U,M)\right)}^{-1}\right]$$

$$=\mathbf{E}\left[1-(1+\mathsf{L}{2}^{{\iota }_{U;S}(U;S)-{\iota }_{U;Y}(U;Y)}{)}^{-1}\right].$$

$$P_e\le \mathbf{P}\{{\iota }_{U;S}(U;S)>\log \mathsf{J}-\gamma \}+\mathbf{P}\{{\iota }_{U;Y}(U;Y)\le \log \mathsf{LJ}+\gamma \}+2^{-\gamma }+e^{-2^{\gamma }}$$

$$\mathbf{E}\left[1-{\left(1+\mathsf{L}{2}^{{\iota }_{U;S}(U;S)-{\iota }_{U;Y}(U;Y)}\right)}^{-1}\right]$$

$$\le \mathbf{P}\{{\iota }_{U;S}(U;S)>\log \mathsf{J}-\gamma \}+\mathbf{P}\{{\iota }_{U;Y}(U;Y)\le \log \mathsf{LJ}+\gamma \}$$

$$+\mathbf{E}\left[1-{\left(1+\mathsf{L}{2}^{{\iota }_{U;S}(U;S)-{\iota }_{U;Y}(U;Y)}\right)}^{-1}|{\iota }_{U;S}(U;S)\le \log \mathsf{J}-\gamma ,{\iota }_{U;Y}(U;Y)>\log \mathsf{LJ}+\gamma \right]$$

$$\le \mathbf{P}\{{\iota }_{U;S}(U;S)>\log \mathsf{J}-\gamma \}+\mathbf{P}\{{\iota }_{U;Y}(U;Y)\le \log \mathsf{LJ}+\gamma \}+2^{-2\gamma }$$

$$<\mathbf{P}\{{\iota }_{U;S}(U;S)>\log \mathsf{J}-\gamma \}+\mathbf{P}\{{\iota }_{U;Y}(U;Y)\le \log \mathsf{LJ}+\gamma \}+2^{-\gamma }+e^{-2^{\gamma }}.$$

$$\mathsf{L}:=\lfloor {\exp }_2\left(nC-\sqrt{nV}{\mathcal{Q}}^{-1}\left(ϵ-\frac{\alpha }{\sqrt{n}}\right)-\frac{1}{2}\log n\right)\rfloor ,$$

$$P_e$$

$$\le \frac{1}{\sqrt{n}}+\mathbf{P}\left\{2^{\log \mathsf{L}+{\iota }_{U^n;S^n}(U^n;S^n)-{\iota }_{U^n;Y^n}(U^n;Y^n)}>\frac{1}{\sqrt{n}}\right\}$$

$$\le \frac{1}{\sqrt{n}}+\mathbf{P}\left\{\frac{1}{\sqrt{n}}\sum _{i=1}^n\left({\iota }_{U;Y}(U_i;Y_i)-{\iota }_{U;S}(U_i;S_i)-C\right)<-\sqrt{V}{\mathcal{Q}}^{-1}\left(ϵ-\frac{\alpha }{\sqrt{n}}\right)\right\}$$

$$\le \frac{1}{\sqrt{n}}+ϵ-\frac{\alpha }{\sqrt{n}}+\frac{\alpha -1}{\sqrt{n}}$$

$$\le ϵ$$

$$P_e\le \mathbf{E}\left[1-1\{\mathsf{d}(X,Z)\le \mathsf{D}\}(1+{\mathsf{L}}^{-1}{2}^{{\iota }_{U;X}(U;X)-{\iota }_{U;Y}(U;Y)}{)}^{-1}\right]$$

$$\mathbf{P}\{\mathsf{d}(X,\hat{Z})>\mathsf{D}\}$$

$$\le 1-\mathbf{P}\{\mathsf{d}(X,Z)\le \mathsf{D}\mathrm{and}U=\hat{U}\}$$

$$\le \mathbf{E}\left[1-1\{\mathsf{d}(X,Z)\le \mathsf{D}\}\mathbf{P}\{(U,M)=(\tilde{U},\tilde{M}{)}_{P_{U|Y}(\cdot |Y)\times {\delta }_M}|M,X,Y,U\}\right]$$

$$\stackrel{(a)}{\le }\mathbf{E}\left[1-1\{\mathsf{d}(X,Z)\le \mathsf{D}\}{\left(1+\frac{d{P}_{U|X}(\cdot |X)\times P_M}{d{P}_{U|Y}(\cdot |Y)\times {\delta }_M}(U,M)\right)}^{-1}\right]$$

Full text

I Introduction

The Poisson functional representation was introduced by Li and El Gamal [1] to prove the strong functional representation lemma: for any pair of random variables $(X,Y)$, there exists a random variable $Z$ independent of $X$ such that $Y$ is a function of $(X,Z)$, and $H(Y|Z)\leq I(X;Y)+\log(I(X;Y)+1)+4$. The lemma is applied to show various one-shot variable-length lossy source coding results, and a simple proof of the asymptotic achievability in the Gelfand-Pinsker theorem [2].

In this paper, we introduce the Poisson matching lemma, which gives a bound on the probability of mismatch between the Poisson functional representations applied on different distributions, and use it to prove one-shot achievability results for various settings, namely channels with state information at the encoder, lossy source coding with side information at the decoder, joint source-channel coding, broadcast channels, distributed lossy source coding, multiple access channels, channel resolvability and wiretap channels. The Poisson matching lemma can replace both the packing and covering lemmas (and generalizations such as the mutual covering lemma) in asymptotic typicality-based proofs. The one-shot bounds in this paper subsume the corresponding asymptotic achievability results by straightforward applications of the law of large numbers.

Various non-asymptotic alternatives to typicality have been proposed, e.g. one-shot packing and covering lemmas [3, 4], stochastic likelihood coder [5], likelihood encoder [6] and random binning [7]. However, these non-asymptotic approaches generally require more complex proofs than their asymptotic counterparts, whereas proofs using the Poisson matching lemma can be even simpler than asymptotic proofs.

Our approach is better than the conventional asymptotic approach using typicality (and previous one-shot results, e.g. [3, 5]), in the following ways:

We can give one-shot bounds stronger than the best known one-shot bounds in many settings discussed in this paper, with the exception of channel coding, multiple access channels, channel resolvability and wiretap channels, which are included for demonstration purposes, where we recover bounds comparable to the best known bounds. 2. 2.

Our proofs work for random variables in general Polish spaces. 3. 3.

To the best of our knowledge, for the achievability in the Gelfand-Pinsker theorem [2] (for channels with state information at the encoder) and the Wyner-Ziv theorem [8, 9] (for lossy source coding with side information at the decoder), our proofs are significantly shorter than all previous proofs (another short proof of the achievability in the Gelfand-Pinsker theorem is given in [1], though it is asymptotic). Using our approach, we can also greatly shorten the proof of the achievability of the dispersion in joint source-channel coding [10]. 4. 4.

Our proofs only use the Poisson matching lemma introduced in this paper, which replaces both the packing and covering lemmas in proofs using typicality. The Poisson matching lemma can also be used to prove a soft covering lemma. Hence the Poisson matching lemma can be the only tool needed to prove a wide range of results in network information theory. 5. 5.

Our analyses usually involve fewer (or no) uses of sub-codebooks and binning. As a result, we can reduce the number of error events and give sharper second-order bounds. For example:

1. (a)

Conventional proofs of the Gelfand-Pinsker theorem involve one sub-codebook, giving an additional error event, whereas we do not use any sub-codebook. 2. (b)

Conventional proofs of the Wyner-Ziv theorem and the Berger-Tung inner bound [11, 12] (for distributed lossy source coding) use binning, giving additional error events, whereas we do not require binning. 3. (c)

Conventional proofs of Marton’s inner bound [13] (for broadcast channels) involve two sub-codebooks, whereas we use only one. 6. 6.

In our approach, the encoders and decoders are characterized using a common framework (the Poisson functional representation), which is noteworthy since the roles of an encoder and a decoder in an operational setting are very different, and their constructions usually have little in common in conventional approaches.

Notation

Throughout this paper, we assume that $\log$ is to base 2 and the entropy $H$ is in bits. We write $\mathrm{exp}_{a}(b)$ for $a^{b}$.

The set of positive integers is denoted as $\mathbb{N}=\{1,2,\ldots\}$. We use the notation: $X_{a}^{b}:=(X_{a},\ldots,X_{b})$, $X^{n}:=X_{1}^{n}$ and $[a:b]:=[a,b]\cap\mathbb{Z}$. The conditional information density is denoted as

[TABLE]

We consider $\iota_{X;Y|Z}(x;y|z)$ to be defined only if $P_{XY|Z=z}\ll P_{X|Z=z}\times P_{Y|Z=z}$.

For discrete $X$, we write the probability mass function as $p_{X}$. For continuous $X$, we write the probability density function as $f_{X}$. For a general random variable $X$ in a measurable space, we write its distribution as $P_{X}$. The uniform distribution over a finite set $S$ is denoted as $\mathrm{Unif}(S)$. The joint distribution of $X_{1},\ldots,X_{n}\stackrel{{\scriptstyle iid}}{{\sim}}P_{X}$ is written as $P_{X}^{\otimes n}$. The degenerate distribution $\mathbf{P}\{X=a\}=1$ is denoted as $\delta_{a}$. The conditional independence of $X$ and $Z$ given $Y$ is denoted as $X\leftrightarrow Y\leftrightarrow Z$.

The Q-function and its inverse are denoted as $\mathcal{Q}(x)$ and $\mathcal{Q}^{-1}(\epsilon)$ respectively. For $V\in\mathbb{R}^{n\times n}$ positive semidefinite, define $\mathcal{Q}^{-1}(V,\epsilon)=\{x\in\mathbb{R}^{n}:\,\mathbf{P}\{X\leq x\}\geq 1-\epsilon\}$ where $X\sim N(0,V)$ and $X\leq x$ denotes entrywise comparison.

We assume that every random variable mentioned in this paper lies in a Polish space with its Borel $\sigma$-algebra, and all functions mentioned (e.g. distortion measures, the function $x(u,s)$ in Theorem 2) are measurable. The Lebesgue measure over $\mathbb{R}$ is denoted as $\lambda$. The Lebesgue measure restricted to the set $S\subseteq\mathbb{R}$ is denoted as $\lambda_{S}$. For two measures $\mu,\nu$ over $\mathcal{X}$ (a Polish space with its Borel $\sigma$-algebra) such that $\nu$ is absolutely continuous with respect to $\mu$ (denoted as $\nu\ll\mu$), the Radon-Nikodym derivative is written as

[TABLE]

If $\nu_{1},\nu_{2}\ll\mu$ (but $\nu_{1}\ll\nu_{2}$ may not hold), we write

[TABLE]

which is [math] if $(d\nu_{1}/d\mu)(x)=0$, and is $\infty$ if $(d\nu_{1}/d\mu)(x)>0$ and $(d\nu_{2}/d\mu)(x)=0$.

The total variation distance between two distributions $P,Q$ over $\mathcal{X}$ is denoted as $\|P-Q\|_{\mathrm{TV}}=\sup_{A\subseteq\mathcal{X}\,\mathrm{measurable}}|P(A)-Q(A)|$.

II Poisson Matching Lemma

We first state the definition of Poisson functional representation in [1], with a different notation that allows the proofs to be written in a simpler and more intuitive manner.

Definition 1** (Poisson functional representation).**

Let $\{\bar{U}_{i},T_{i}\}_{i\in\mathbb{N}}$ be the points of a Poisson process with intensity measure $\mu\times\lambda_{\mathbb{R}_{\geq 0}}$ on $\mathcal{U}\times\mathbb{R}_{\geq 0}$ (where $\mathcal{U}$ is a Polish space with its Borel $\sigma$-algebra, and $\mu$ is $\sigma$-finite). For $P\ll\mu$ a probability measure over $\mathcal{U}$, define

[TABLE]

where

[TABLE]

with arbitrary tie-breaking (a tie occurs with probability 0). We omit $\{\bar{U}_{i},T_{i}\}_{i\in\mathbb{N}}$ and only write $\tilde{U}_{P}$ if the Poisson process is clear from the context. If the Poisson process is $\{\bar{X}_{i},T_{i}\}_{i\in\mathbb{N}}$ instead of $\{\bar{U}_{i},T_{i}\}_{i\in\mathbb{N}}$, then the Poisson functional representation is likewise denoted as $\tilde{X}_{P}$. If $\bar{U}_{i}=(\bar{X}_{i},\bar{Y}_{i})$ is multivariate, and $P$ is a distribution over $\mathcal{X}\times\mathcal{Y}$, the Poisson functional representation is denoted as $(\tilde{X},\tilde{Y})_{P}$. We write its components as $(\tilde{X},\tilde{Y})_{P}=(\tilde{X}_{P},\tilde{Y}_{P})$.

Note that while $dP/d\mu$ is only uniquely defined up to a $\mu$-null set, changing the value of $dP/d\mu$ on a $\mu$-null set will only affect the values of $\tilde{U}_{P}$ on a null set with respect to the distribution of $\{\bar{U}_{i},T_{i}\}_{i\in\mathbb{N}}$, since the probability that there exists $\bar{U}_{i}$ on that $\mu$-null set is zero. Therefore $\tilde{U}_{P}$ is uniquely defined up to a null set.

By the mapping theorem [14, 15] (also see Appendix A of [1]), we have $\tilde{U}_{P}\sim P$. This is termed Poisson functional representation in [1] since it can be regarded as a construction for the functional representation lemma [16]. Consider the distribution $P_{U,X}$. Let $\{\bar{U}_{i},T_{i}\}_{i\in\mathbb{N}}$ be the points of a Poisson process with intensity measure $P_{U}\times\lambda_{\mathbb{R}_{\geq 0}}$, $X\sim P_{X}$ independent of the process, and $U:=\tilde{U}_{P_{U|X}(\cdot|X)}$. Then $(U,X)\sim P_{U,X}$. Hence we can express $U$ as a function of $X$ and $\{\bar{U}_{i},T_{i}\}$ (which is independent of $X$). This fact will be used repeatedly throughout the proofs in this paper.

For two different distributions $P$ and $Q$, $\tilde{U}_{P}$ and $\tilde{U}_{Q}$ are coupled in such a way that $\tilde{U}_{P}=\tilde{U}_{Q}$ occurs with a probability that can be bounded in terms of $dP/dQ$. We now present the core lemma of this paper. The proof is given in Appendix -A.

Lemma 1** (Poisson matching lemma).**

Let $\{\bar{U}_{i},T_{i}\}_{i\in\mathbb{N}}$ be the points of a Poisson process with intensity measure $\mu\times\lambda_{\mathbb{R}_{\geq 0}}$, and $P,Q$ be probability measures on $\mathcal{U}$ with $P,Q\ll\mu$. Then we have the following almost surely:

[TABLE]

where we write $(dP/dQ)(u)=(dP/d\mu)(u)/((dQ/d\mu)(u))$ as in (1) (we do not require $P\ll Q$). The right hand side of (2) is considered to be 1 if $(dP/d\mu)(\tilde{U}_{P})>0$ and $(dQ/d\mu)(\tilde{U}_{P})=0$.

The exact expression for the left hand side of (2) is in (16).

We usually do not apply the Poisson matching lemma on fixed $P,Q$, but rather on conditional distributions. The following conditional version of the Poisson matching lemma follows directly from applying the lemma on $(P,Q)\leftarrow(P_{U|X}(\cdot|X),Q_{U|Y}(\cdot|Y))$. The proof is given in Appendix -B for the sake of completeness.

Lemma 2** (Conditional Poisson matching lemma).**

Fix a distribution $P_{X,U,Y}$ and a probability kernel $Q_{U|Y}$ (that is not necessarily $P_{U|Y}$) satisfying $P_{U|X}(\cdot|X),Q_{U|Y}(\cdot|Y)\ll\mu$ almost surely. Let $X\sim P_{X}$, and $\{\bar{U}_{i},T_{i}\}_{i\in\mathbb{N}}$ be the points of a Poisson process with intensity measure $\mu\times\lambda_{\mathbb{R}_{\geq 0}}$ independent of $X$. Let $U=\tilde{U}_{P_{U|X}(\cdot|X)}$ and $Y|(X,U,\{\bar{U}_{i},T_{i}\}_{i})\sim P_{Y|X,U}(\cdot|X,U)$ (note that $(X,U,Y)\sim P_{X,U,Y}$ and $Y\leftrightarrow(X,U)\leftrightarrow\{\bar{U}_{i},T_{i}\}_{i}$). Then we have the following almost surely:

[TABLE]

The condition that $P_{U|X}(\cdot|X),Q_{U|Y}(\cdot|Y)\ll\mu$ almost surely is satisfied, for example, when $\mu=P_{U}$, $Q_{U|Y}=P_{U|Y}$, $P_{UX}\ll P_{U}\times P_{X}$ and $P_{UY}\ll P_{U}\times P_{Y}$. Note that since $X{\perp\!\!\!\perp}\{\bar{U}_{i},T_{i}\}_{i}$, we have $\tilde{U}_{P_{U|X}(\cdot|X)}|X\sim P_{U|X}$, whereas $Y$ may not be independent of $\{\bar{U}_{i},T_{i}\}_{i}$, so $\tilde{U}_{Q_{U|Y}(\cdot|Y)}$ may not follow the conditional distribution $Q_{U|Y}$.

III One-shot Channel Coding

To demonstrate the application of the Poisson matching lemma, we apply it to recover a bound for one-shot channel coding in [5] (with a slight penalty of having $\mathsf{L}$ instead of $\mathsf{L}-1$). Upon observing $M\sim\mathrm{Unif}[1:\mathsf{L}]$, the encoder produces $X$, which is sent through the channel $P_{Y|X}$. The decoder observes $Y$ and recovers $\hat{M}$ with error probability $P_{e}=\mathbf{P}\{M\neq\hat{M}\}$.

Proposition 1**.**

Fix any $P_{X}$. There exists a code for the channel $P_{Y|X}$, with message $M\sim\mathrm{Unif}[1:\mathsf{L}]$, with average error probability

[TABLE]

if $P_{XY}\ll P_{X}\times P_{Y}$.

Proof:

Let $\{(\bar{X}_{i},\bar{M}_{i}),T_{i}\}_{i\in\mathbb{N}}$ be the points of a Poisson process with intensity measure $P_{X}\times P_{M}\times\lambda_{\mathbb{R}_{\geq 0}}$ (where $P_{M}$ is $\mathrm{Unif}[1:\mathsf{L}]$) independent of $M$. The encoding function is $m\mapsto\tilde{X}_{P_{X}\times\delta_{m}}$ (i.e., $X=\tilde{X}_{P_{X}\times\delta_{M}}$), and the decoding function is $y\mapsto\tilde{M}_{P_{X|Y}(\cdot|y)\times P_{M}}$ (i.e., $\hat{M}=\tilde{M}_{P_{X|Y}(\cdot|Y)\times P_{M}}$). Note that the encoding and decoding functions also depend on the common randomness $\{(\bar{X}_{i},\bar{M}_{i}),T_{i}\}_{i\in\mathbb{N}}$, which will be fixed later. We have $(M,X,Y)\sim P_{M}\times P_{X}P_{Y|X}$.

[TABLE]

where (a) is by the conditional Poisson matching lemma (Lemma 2) on $(X,U,Y,Q_{U|Y})\leftarrow(M,(X,M),Y,P_{X|Y}\times P_{M})$ (note that $P_{X,M|M}=P_{X}\times\delta_{M}$). Therefore there exists a fixed $\{(\bar{x}_{i},\bar{m}_{i}),t_{i}\}_{i\in\mathbb{N}}$ such that conditioned on $\{(\bar{X}_{i},\bar{M}_{i}),T_{i}\}_{i\in\mathbb{N}}=\{(\bar{x}_{i},\bar{m}_{i}),t_{i}\}_{i\in\mathbb{N}}$, the average probability of error is bounded by $\mathbf{E}\left[1-(1+\mathsf{L}2^{-\iota_{X;Y}(X;Y)})^{-1}\right]$. ∎

Compared to the scheme in [5], we use the Poisson process $\{(\bar{X}_{i},\bar{M}_{i}),T_{i}\}$ to create a codebook, instead of the conventional i.i.d. random codebook in [5]. While the codewords for different $m$’s are still i.i.d., we attach a bias $T_{i}$ to each codeword. Our scheme does not use a stochastic decoder as in [5], but rather a biased maximum likelihood decoder $\tilde{M}_{P_{X|Y}(\cdot|y)\times P_{M}}=\bar{M}_{K}$ where $K=\arg\max_{i}T_{i}^{-1}(dP_{X|Y}(\cdot|y)/dP_{X})(\bar{X}_{i})$. In the following sections, we will demonstrate how our approach can lead to simpler proofs and sharper bounds compared to [5].

Using the generalized Poisson matching lemma that will be introduced in Section VII, we can prove the following bound. The proof is in Appendix -C.

Theorem 1**.**

Fix any $P_{X}$. There exists a code for the channel $P_{Y|X}$, with message $M\sim\mathrm{Unif}[1:\mathsf{L}]$, with average error probability

[TABLE]

if $P_{XY}\ll P_{X}\times P_{Y}$.

Compare this to the dependence testing bound [17]:

[TABLE]

Theorem 1 is at least as strong (with a slight penalty of having $(\mathsf{L}+1)/2$ instead of $(\mathsf{L}-1)/2$) since

[TABLE]

Remark 1*.*

Apart from the dependence testing bound [17], there are other one-shot bounds for channel coding such as the random-coding union (RCU) bound and the $\kappa\beta$ bound in [17], which are tighter in certain situations (e.g. the RCU bound is suitable for error exponent analysis). The technique introduced in this paper is suitable for first and second order analysis, but does not seem to give tight error exponent bounds.

IV One-shot Coding for Channels with State Information at the Encoder

The one-shot coding setting for a channel with state information at the encoder is described as follows. Upon observing $M\sim\mathrm{Unif}[1:\mathsf{L}]$ and $S\sim P_{S}$, the encoder produces $X$, which is sent through the channel $P_{Y|X,S}$ with state $S$. The decoder observes $Y$ and recovers $\hat{M}$ with error probability $P_{e}=\mathbf{P}\{M\neq\hat{M}\}$.

We show a one-shot version of the Gelfand-Pinsker theorem [2]. This is the first one-shot bound attaining the best known second order result in [18] (which considers a finite-blocklength, not one-shot scenario). Our bound is stronger than the one-shot bounds in [3, 5, 19] (in the second order), and significantly simpler to state and prove than all the aforementioned results. Unlike previous approaches, our proof does not require sub-codebooks.

Theorem 2**.**

Fix any $P_{U|S}$ and function $x:\mathcal{U}\times\mathcal{S}\to\mathcal{X}$. There exists a code for the channel $P_{Y|X,S}$ with state distribution $P_{S}$ with message $M\sim\mathrm{Unif}[1:\mathsf{L}]$, with error probability

[TABLE]

if $P_{US}\ll P_{U}\times P_{S}$ and $P_{UY}\ll P_{U}\times P_{Y}$, where $(S,U,X,Y)\sim P_{S}P_{U|S}\delta_{x(U,S)}P_{Y|X,S}$.

Proof:

Let $\{(\bar{U}_{i},\bar{M}_{i}),T_{i}\}_{i\in\mathbb{N}}$ be the points of a Poisson process with intensity measure $P_{U}\times P_{M}\times\lambda_{\mathbb{R}_{\geq 0}}$ independent of $M,S$. The encoding function is $(m,s)\mapsto x(\tilde{U}_{P_{U|S}(\cdot|s)\times\delta_{m}},s)$ (let $U=\tilde{U}_{P_{U|S}(\cdot|S)\times\delta_{M}}$, $X=x(U,S)$), and the decoding function is $y\mapsto\tilde{M}_{P_{U|Y}(\cdot|y)\times P_{M}}$ (i.e., $\hat{M}=\tilde{M}_{P_{U|Y}(\cdot|Y)\times P_{M}}$). Note that $(M,S,U,X,Y)\sim P_{M}\times P_{S}P_{U|S}\delta_{x(U,S)}P_{Y|X,S}$. We have

[TABLE]

where (a) is by the conditional Poisson matching lemma on $((M,S),\,(U,M),\,Y,\,P_{U|Y}\times P_{M})$ (note that $P_{U,M|M,S}=P_{U|S}\times\delta_{M}$). Therefore there exists a fixed $\{(\bar{u}_{i},\bar{m}_{i}),t_{i}\}_{i\in\mathbb{N}}$ attaining the desired bound. ∎

Compared to Theorem 3 in [3]:

[TABLE]

for any $\gamma>0$, $\mathsf{J}\in\mathbb{N}$, our result is strictly stronger since

[TABLE]

This is due to the fact that the Poisson matching lemma simultaneously replaces both the covering and the packing lemma, resulting in only one error event.

Next, we prove a second-order result. Fix $\epsilon>0$. Let $C:=I(U;Y)-I(U;S)$, $V:=\mathrm{Var}[\iota_{U;S}(U;S)-\iota_{U;Y}(U;Y)]$. We apply Theorem 2 on $n$ uses of the memoryless channel with i.i.d. state sequence $S^{n}=(S_{1},\ldots,S_{n})$, and

[TABLE]

where $\alpha$ is a constant that depends on $P_{S,U,Y}$. For $n>\alpha^{2}\epsilon^{-2}$, by the Berry-Esseen theorem [20, 21, 22], we have

[TABLE]

if we let $\alpha-1$ be the constant given by the Berry-Esseen theorem. This coincides with the best known second order result in [18], which is stronger than the second order results implied by [3, 5, 19]. We bound $\iota_{U;S}(U;S)-\iota_{U;Y}(U;Y)$ as a single quantity, instead of bounding the two terms separately as in [3, 5, 19], resulting in a sharper second order bound.

V One-shot Lossy Source Coding with Side Information at the Decoder

The one-shot lossy source coding setting with side information at the decoder is described as follows. Upon observing $X\sim P_{X}$, the encoder produces $M\in[1:\mathsf{L}]$. The decoder observes $M$ and $Y\sim P_{Y|X}$ and recovers $\hat{Z}\in\mathcal{Z}$ with probability of excess distortion $P_{e}=\mathbf{P}\{\mathsf{d}(X,\hat{Z})>\mathsf{D}\}$, where $\mathsf{d}:\mathcal{X}\times\mathcal{Z}\to\mathbb{R}_{\geq 0}$ is a distortion measure.

We show a one-shot version of the Wyner-Ziv theorem [8, 9]. Our bound is stronger than those in [3, 19], and significantly simpler to state and prove. Unlike previous approaches, our proof does not require binning.

Theorem 3**.**

Fix any $P_{U|X}$ and function $z:\mathcal{U}\times\mathcal{Y}\to\mathcal{Z}$. There exists a code for lossy source coding with source distribution $P_{X}$, side information at the decoder given by $P_{Y|X}$, and message size $\mathsf{L}$, with probability of excess distortion

[TABLE]

if $P_{UX}\ll P_{U}\times P_{X}$ and $P_{UY}\ll P_{U}\times P_{Y}$, where $(X,Y,U,Z)\sim P_{X}P_{Y|X}P_{U|X}\delta_{z(U,Y)}$.

Proof:

Let $\{(\bar{U}_{i},\bar{M}_{i}),T_{i}\}_{i\in\mathbb{N}}$ be the points of a Poisson process with intensity measure $P_{U}\times P_{M}\times\lambda_{\mathbb{R}_{\geq 0}}$ independent of $X$, where $P_{M}$ is $\mathrm{Unif}[1:\mathsf{L}]$. The encoding function is $x\mapsto\tilde{M}_{P_{U|X}(\cdot|x)\times P_{M}}$ (i.e., $M=\tilde{M}_{P_{U|X}(\cdot|X)\times P_{M}}$), and the decoding function is $(m,y)\mapsto z(\tilde{U}_{P_{U|Y}(\cdot|y)\times\delta_{m}},y)$ (let $\hat{U}=\tilde{U}_{P_{U|Y}(\cdot|Y)\times\delta_{M}}$, $\hat{Z}=z(\hat{U},Y)$). Also define $U=\tilde{U}_{P_{U|X}(\cdot|X)\times P_{M}}$, $Z=z(U,Y)$. Note that $(M,X,Y,U,Z)\sim P_{M}\times P_{X}P_{Y|X}P_{U|X}\delta_{z(U,Y)}$. We have

[TABLE]

where (a) is by the conditional Poisson matching lemma on $(X,\,(U,M),\,(M,Y),\,P_{U|Y}\times\delta_{M})$ (note that $P_{U,M|X}=P_{U|X}\times P_{M}$). Therefore there exists a fixed $\{(\bar{u}_{i},\bar{m}_{i}),t_{i}\}_{i\in\mathbb{N}}$ attaining the desired bound. ∎

This reduces to lossy source coding (without side information) when $Y=\emptyset$. Note that the encoder is designed in the same way with or without side information. An encoder for lossy source coding is sufficient to achieve the bound in Theorem 3 even when side information is present. Binning is not required at the encoder.

Similar to the case in Section IV, it can be checked that our bound is stronger than that in Theorem 2 in [3]. Compared to Corollary 9 in [19]:

[TABLE]

for any $\gamma_{\mathrm{p}},\gamma_{\mathrm{c}}>0$, $\mathsf{J}\in\mathbb{N}$, our result is stronger since

[TABLE]

where the last inequality is due to

[TABLE]

by the AM-GM inequality for $a,b\geq 0$, $a+b\leq 1$ (since the right hand side of (3) $\leq 1$ for it to be meaningful). We bound $\iota_{U;X}(U;X)-\iota_{U;Y}(U;Y)$ as a single quantity, instead of bounding the two terms separately, resulting in a sharper bound.

VI One-shot Joint Source-Channel Coding

The one-shot joint source-channel coding setting is described as follows. Upon observing the source symbol $W\sim P_{W}$, the encoder produces $X\in\mathcal{X}$, which is sent through the channel $P_{Y|X}$. The decoder observes $Y$ and recovers $\hat{Z}\in\mathcal{Z}$ with probability of excess distortion $P_{e}=\mathbf{P}\{\mathsf{d}(W,\hat{Z})>\mathsf{D}\}$, where $\mathsf{d}:\mathcal{W}\times\mathcal{Z}\to\mathbb{R}_{\geq 0}$ is a distortion measure.

We show a one-shot joint source-channel coding result that achieves the optimal dispersion in [10].

Theorem 4**.**

Fix any $P_{X}$ and $P_{Z}$. There exists a code for the source distribution $P_{W}$ and channel $P_{Y|X}$, with probability of excess distortion

[TABLE]

if $P_{XY}\ll P_{X}\times P_{Y}$, where $(W,X,Y)\sim P_{W}\times P_{X}P_{Y|X}$, and $\mathcal{B}_{\mathsf{D}}(w):=\{z:\,\mathsf{d}(w,z)\leq\mathsf{D}\}$.

Proof:

Let $\{(\bar{X}_{i},\bar{Z}_{i}),T_{i}\}_{i\in\mathbb{N}}$ be the points of a Poisson process with intensity measure $P_{X}\times P_{Z}\times\lambda_{\mathbb{R}_{\geq 0}}$ independent of $W$. Let $\rho(w):=P_{Z}(\mathcal{B}_{\mathsf{D}}(w))$. Let $P_{\check{Z}|W}$ be defined as

[TABLE]

The encoding function is $w\mapsto\tilde{X}_{P_{X}\times P_{\check{Z}|W}(\cdot|w)}$ (i.e., $X=\tilde{X}_{P_{X}\times P_{\check{Z}|W}(\cdot|W)}$). The decoding function is $y\mapsto\tilde{Z}_{P_{X|Y}(\cdot|y)\times P_{Z}}$ (i.e., $\hat{Z}=\tilde{Z}_{P_{X|Y}(\cdot|Y)\times P_{Z}}$). Also define $\check{Z}=\tilde{Z}_{P_{X}\times P_{\check{Z}|W}(\cdot|W)}$. We have $(X,Y,W,\check{Z})\sim P_{X}P_{Y|X}\times P_{W}P_{\check{Z}|W}$.

[TABLE]

where (a) is by the conditional Poisson matching lemma on $(W,\,(X,\check{Z}),\,Y,\,P_{X|Y}\times P_{Z})$ (note that $P_{X,\check{Z}|W}=P_{X}\times P_{\check{Z}|W}$). Therefore there exists a fixed $\{(\bar{x}_{i},\bar{z}_{i}),t_{i}\}_{i\in\mathbb{N}}$ attaining the desired bound. ∎

Compare this to Theorem 7 in [10]:

[TABLE]

for any $P_{J|W}$, $J\in\mathbb{N}$. While neither of the bounds implies the other, our bound is at least within a factor of 2 from (4), since

[TABLE]

However, (4) does not imply a bound that is within a constant factor from our bound. Theorem 8 in [10] is obtained by substituting $J=\lfloor\gamma/P_{Z}(\mathcal{B}_{\mathsf{D}}(W))\rfloor$ in (4):

[TABLE]

which is strictly weaker than our bound with an unbounded multiplicative gap $\gamma$ (that tends to $\infty$ when the bound tends to 0). Hence our bound is stronger than Theorem 7 and 8 in [10] (ignoring constant multiplicative gaps). Also our proof is significantly shorter than that of Theorem 7 in [10].

Please refer to Appendix -D for the proof that Theorem 4 achieves the optimal dispersion.

VII Poisson Matching Lemma Beyond the First Index

The Poisson functional representation concerns the point with the smallest $T_{i}((dP/d\mu)(\bar{U}_{i}))^{-1}$. We can generalize it to obtain a sequence ordered in ascending order of $T_{i}((dP/d\mu)(\bar{U}_{i}))^{-1}$.

Definition 2** (Mapped Poisson process).**

Let $\{\bar{U}_{i},T_{i}\}_{i\in\mathbb{N}}$ be the points of a Poisson process with intensity measure $\mu\times\lambda_{\mathbb{R}_{\geq 0}}$ on $\mathcal{U}\times\mathbb{R}_{\geq 0}$ (where $\mathcal{U}$ is a Polish space with its Borel $\sigma$-algebra, and $\mu$ is $\sigma$-finite). For $P\ll\mu$ a probability measure over $\mathcal{U}$, let $i_{P,1},i_{P,2},\ldots\in\mathbb{N}$ be a sequence of distinct integers such that $\bigcup_{j=1}^{\infty}\{i_{P,j}\}=\{i:\,(dP/d\mu)(\bar{U}_{i})>0\}$ and $\{T_{i_{P,j}}((dP/d\mu)(\bar{U}_{i_{P,j}}))^{-1}\}_{j\in\mathbb{N}}$ is sorted in ascending order with arbitrary tie-breaking (a tie occurs with probability 0). For $j\in\mathbb{N},\,u\in\mathcal{U}$, define the *mapped Poisson process with respect to $P$ *as

[TABLE]

where

[TABLE]

[TABLE]

For $P,Q\ll\mu$ probability measures over $\mathcal{U}$, define $i_{P,1},i_{P,2},\ldots\in\mathbb{N}$ and $i_{Q,1},i_{Q,2},\ldots\in\mathbb{N}$ as above. Define

[TABLE]

where the minimum is $\infty$ if such $k$ does not exist. We omit $\{\bar{U}_{i},T_{i}\}_{i\in\mathbb{N}}$ and only write $\tilde{U}_{P}(j)$, $\tilde{T}_{P}(j)$, $\Upsilon_{P\|Q}(j)$ if the Poisson process is clear from the context. Note that, with probability 1, we have either $\tilde{U}_{Q}(\Upsilon_{P\|Q}(j))=\tilde{U}_{P}(j)$ or $\Upsilon_{P\|Q}(j)=\infty$. Also, for any $j,k\in\mathbb{N}$, $\Upsilon_{P\|Q}(j)=k\Leftrightarrow\Upsilon_{Q\|P}(k)=j$. Loosely speaking, $\Upsilon_{P\|Q}(j)$ can be regarded as “$\tilde{U}_{Q}^{-1}(\tilde{U}_{P}(j))$” (if there are no atoms in $\mu$), i.e., finding the $j$-th point in the mapped Poisson process w.r.t. $P$, then finding its index in the mapped Poisson process w.r.t. $Q$.

While $dP/d\mu$ is only uniquely defined up to a $\mu$-null set, changing the value of $dP/d\mu$ on a $\mu$-null set will only affect the values of $\{\tilde{U}_{P}(j),\,\tilde{T}_{P}(j)\}_{j\in\mathbb{N}}$ on a null set with respect to the distribution of $\{\bar{U}_{i},T_{i}\}_{i\in\mathbb{N}}$, since the probability that there exists $\bar{U}_{i}$ in that $\mu$-null set is zero. Therefore $\{\tilde{U}_{P}(j),\,\tilde{T}_{P}(j)\}_{j\in\mathbb{N}}$ is uniquely defined up to a null set. The same is true for $\Upsilon_{P\|Q}(j)$.

By the mapping theorem [14, 15] (also see Appendix A of [1]),

[TABLE]

is a Poisson process with intensity measure $P\times\lambda_{\mathbb{R}_{\geq 0}}$. Hence

[TABLE]

We present a generalized Poisson matching lemma concerning the indices beyond the first. The proof is given in Appendix -A.

Lemma 3** (Generalized Poisson matching lemma).**

Let $\{\bar{U}_{i},T_{i}\}_{i\in\mathbb{N}}$ be the points of a Poisson process with intensity measure $\mu\times\lambda_{\mathbb{R}_{\geq 0}}$ on $\mathcal{U}\times\mathbb{R}_{\geq 0}$, and $P,Q$ be probability measures over $\mathcal{U}$ with $P,Q\ll\mu$. Fix any $j\in\mathbb{N}$. Then we have the following almost surely:

[TABLE]

where we write $(dP/dQ)(u)=(dP/d\mu)(u)/((dQ/d\mu)(u))$ as in (1) (we do not require $P\ll Q$). As a result, we have the following almost surely: for all $k\in\mathbb{N}$,

[TABLE]

For $k=1$, this can be slightly strengthened to

[TABLE]

For $j=1$, this can be slightly strengthened to: for all $k\in\mathbb{N}$,

[TABLE]

The exact distribution of $\Upsilon_{P\|Q}(j)$ is given in (15).

Similar to Lemma 2, we can state a conditional version of the generalized Poisson matching lemma. The proof follows the same logic as Lemma 2 and is omitted.

Lemma 4** (Conditional generalized Poisson matching lemma).**

Fix a distribution $P_{X,J,U,Y}$ and a probability kernel $Q_{U|Y}$, satisfying $J\in\mathbb{N}$ and $P_{U|X,J}(\cdot|X,J),Q_{U|Y}(\cdot|Y)\ll\mu$ almost surely. Let $(X,J)\sim P_{X,J}$, and $\{\bar{U}_{i},T_{i}\}_{i\in\mathbb{N}}$ be the points of a Poisson process with intensity measure $\mu\times\lambda_{\mathbb{R}_{\geq 0}}$ independent of $(X,J)$. Let $U=\tilde{U}_{P_{U|X,J}(\cdot|X,J)}(J)$ and $Y|(X,J,U,\{\bar{U}_{i},T_{i}\}_{i})\sim P_{Y|X,J,U}(\cdot|X,J,U)$ (note that $(X,J,U,Y)\sim P_{X,J,U,Y}$ and $Y\leftrightarrow(X,J,U)\leftrightarrow\{\bar{U}_{i},T_{i}\}_{i}$). Then we have the following almost surely:

[TABLE]

and for all $k\in\mathbb{N}$,

[TABLE]

and

[TABLE]

If $J=1$ almost surely, then we also have the following almost surely: for all $k\in\mathbb{N}$,

[TABLE]

Remark 2*.*

We can use the generalized Poisson matching lemma to extend Proposition 1 to the list decoding setting with fixed list size $\mathsf{J}$. The decoder outputs the list $\{\tilde{M}_{P_{X|Y}(\cdot|Y)\times P_{M}}(j)\}_{j\in[1:\mathsf{J}]}$. The error event becomes $(X,M)\notin\{(\tilde{X},\tilde{M})_{P_{X|Y}(\cdot|Y)\times P_{M}}(j)\}_{j\in[1:\mathsf{J}]}$. The probability of error is bounded by $\mathbf{E}\left[(1-(1+\mathsf{L}2^{-\iota_{X;Y}(X;Y)})^{-1})^{\mathsf{J}}\right]$.

VIII One-shot Coding for Broadcast Channels and Mutual Covering

The one-shot coding setting for the broadcast channel with common message is described as follows. Upon observing three independent messages $M_{j}\sim\mathrm{Unif}[1:\mathsf{L}_{j}]$, $j=0,1,2$, the encoder produces $X$, which is sent through the broadcast channel $P_{Y_{1},Y_{2}|X}$. Decoder $j$ observes $Y_{j}$ and recovers $\hat{M}_{0j}$ and $\hat{M}_{j}$ ($j=1,2$). The error probability is $P_{e}=\mathbf{P}\{(M_{0},M_{0},M_{1},M_{2})\neq(\hat{M}_{01},\hat{M}_{02},\hat{M}_{1},\hat{M}_{2})\}$.

We show a one-shot version of the inner bound in [23, Theorem 5] (which is shown to be equivalent to [24, Theorem 1] in [25]). The proof is given in Appendix -F.

Theorem 5**.**

Fix any $P_{U_{0},U_{1},U_{2}}$ and function $x:\mathcal{U}_{0}\times\mathcal{U}_{1}\times\mathcal{U}_{2}\to\mathcal{X}$. For any $\mathsf{J},\mathsf{K}_{1},\mathsf{K}_{2}\in\mathbb{N}$, there exists a code for the broadcast channel $P_{Y_{1},Y_{2}|X}$ for independent messages $M_{j}\sim\mathrm{Unif}[1:\mathsf{L}_{j}]$, $j=0,1,2$, with the error probability bounded by

[TABLE]

if all the information density terms are defined almost surely, where

[TABLE]

As a result, for $\gamma>0$,

[TABLE]

The logarithmic terms $A$ and $B$ (or the last term in (6)) result in an $O(n^{-1}\log n)$ penalty on the rate in the finite blocklength regime, and do not affect the second order result. Ignoring the last term in (6), the error event in (6) is a strict subset of those in [5, eqn (32)] and [4, eqn (49)]. This is because the error event in [5] is a superset of (6) by Fourier-Motzkin elimination on $\mathsf{J}_{2}$ in the error event in [5], but the reverse is not true since Fourier-Motzkin elimination only guarantees the existence of a random variable for $\mathsf{J}_{2}$ (that depends on the information density terms) satisfying the bounds, but $\mathsf{J}_{2}$ must be a constant since it is a parameter of the code construction in [5].

Theorem 5 gives the following second order bound. Consider $n$ independent channel uses. Let $\mathsf{L}_{a}=2^{nR_{a}}$ for $a=0,1,2$. By the multi-dimensional Berry-Esseen theorem [26] (using the notation in [5]), we have $P_{e}\leq\epsilon$ if there exists $\bar{R},\hat{R}_{1},\hat{R}_{2}\geq 0$ such that

[TABLE]

if $n>\beta^{2}\epsilon^{-2}$, where $\beta$ is a constant that depends on $P_{U_{0},U_{1},U_{2},Y_{1},Y_{2}}$, and $\tilde{R}_{0}=R_{0}+\hat{R}_{1}+\hat{R}_{2}$, $\tilde{R}_{a}=R_{a}-\hat{R}_{a}$ for $a=1,2$, and

[TABLE]

To demonstrate the use of the generalized Poisson matching lemma in place of the mutual covering lemma, we prove a one-shot version of Marton’s inner bound without common message [13] (i.e., $\mathsf{L}_{0}=1$). Our bound is stronger than that in [3] in the sense that our bound implies [3] (with a slight penalty of having $2^{1-\gamma}+2^{-2\gamma}$ instead of $2^{1-\gamma}+e^{-2^{\gamma}}$), but [3] does not imply our bound. We also note that a finite-blocklength bound is given in [7]. Nevertheless, the analysis in [7] only works for discrete auxiliary random variables $U_{1},U_{2}$, and does not appear to yield a one-shot bound due to the use of typical sequences.

In the conventional mutual covering approach in [5, 4], sub-codebooks for both $U_{1}$ and $U_{2}$ are generated, whereas in our approach we generate a sub-codebook only for $U_{1}$, and the codebook of $U_{2}$ adapts to the sub-codebook automatically, eliminating the need for a sub-codebook for $U_{2}$.

Theorem 6**.**

Fix any $P_{U_{1},U_{2}}$ and function $x:\mathcal{U}_{1}\times\mathcal{U}_{2}\to\mathcal{X}$. For any $\mathsf{J}\in\mathbb{N}$, there exists a code for the broadcast channel $P_{Y_{1},Y_{2}|X}$ for independent private messages $M_{j}\sim\mathrm{Unif}[1:\mathsf{L}_{j}]$, $j=1,2$, with the error probability bounded by

[TABLE]

if all the information density terms are defined, where $(U_{1},U_{2},X,Y_{1},Y_{2})\sim P_{U_{1}U_{2}}\delta_{x(U_{1},U_{2})}P_{Y_{1},Y_{2}|X}$.

Proof:

Let $\{(\bar{U}_{1,i},\bar{M}_{1,i}),T_{1,i}\}_{i\in\mathbb{N}}$, $\{(\bar{U}_{2,i},\bar{M}_{2,i}),T_{2,i}\}_{i\in\mathbb{N}}$ be two independent Poisson processes with intensity measures $P_{U_{1}}\times P_{M_{1}}\times\lambda_{\mathbb{R}_{\geq 0}}$ and $P_{U_{2}}\times P_{M_{2}}\times\lambda_{\mathbb{R}_{\geq 0}}$ respectively, independent of $M_{1},M_{2}$.

The encoder would generate $X$ such that

[TABLE]

where $P_{K}=\mathrm{Unif}[1:\mathsf{J}]$, and $\{\check{U}_{1j}\}_{j\in[1:\mathsf{J}]}\in\mathcal{U}_{1}^{\mathsf{J}}$ is an intermediate list (which can be regarded as a sub-codebook). The term $P_{U_{1}}^{\otimes\mathsf{J}}\delta_{\check{U}_{1K}}$ in (7) means that $\{\check{U}_{1j}\}_{j}$ are i.i.d. $P_{U_{1}}$, and $U_{1}=\check{U}_{1K}$. To accomplish this, the encoder computes $\check{U}_{1j}=(\tilde{U}_{1})_{P_{U_{1}}\times\delta_{M_{1}}}(j)$ for $j=1,\ldots,\mathsf{J}$ (which Poisson process we are referring to can be deduced from whether we are discussing $U_{1}$ or $U_{2}$), $U_{2}=(\tilde{U}_{2})_{\mathsf{J}^{-1}\sum_{j=1}^{\mathsf{J}}P_{U_{2}|U_{1}}(\cdot|\check{U}_{1j})\times\delta_{M_{2}}}$, and $(K,U_{1})|(\{\check{U}_{1j}\}_{j},U_{2})\sim P_{K,U_{1}|\{\check{U}_{1j}\}_{j},U_{2}}$ (where $P_{K,U_{1}|\{\check{U}_{1j}\}_{j},U_{2}}$ is derived from (7)), and outputs $X=x(U_{1},U_{2})$. It can be verified that (7) is satisfied.

The decoding functions are $\hat{M}_{1}=(\tilde{M}_{1})_{P_{U_{1}|Y_{1}}(\cdot|Y_{1})\times P_{M_{1}}}$, $\hat{M}_{2}=(\tilde{M}_{2})_{P_{U_{2}|Y_{2}}(\cdot|Y_{2})\times P_{M_{2}}}$. We have the following almost surely:

[TABLE]

where (a) is by $(U_{2},Y_{2})\leftrightarrow(U_{1},Y_{1},M_{1},K)\leftrightarrow\{(\bar{U}_{1,i},\bar{M}_{1,i}),T_{1,i}\}_{i}$ (see Figure 1 middle), and (b) is by the conditional generalized Poisson matching lemma on $(X,J,U,Y,Q_{U|Y})\leftarrow(M_{1},\,K,\,(U_{1},M_{1}),\,Y_{1},\,P_{U_{1}|Y_{1}}\times P_{M_{1}})$, since $P_{U_{1},M_{1}|M_{1},K}=P_{U_{1}}\times\delta_{M_{1}}$, $(M_{1},K)\perp\!\!\!\perp\{(\bar{U}_{1,i},\bar{M}_{1,i}),T_{1,i}\}_{i}$, and $Y_{1}\leftrightarrow(U_{1},M_{1},K)\leftrightarrow\{(\bar{U}_{1,i},\bar{M}_{1,i}),T_{1,i}\}_{i}$, which can be deduced from (7) and $\check{U}_{1j}=(\tilde{U}_{1})_{P_{U_{1}}\times\delta_{M_{1}}}(j)$ (see Figure 1 middle).

Also, almost surely,

[TABLE]

where (a) is by $Y_{1}\leftrightarrow(U_{1},U_{2},Y_{2},M_{2})\leftrightarrow\{(\bar{U}_{2,i},\bar{M}_{2,i}),T_{2,i}\}_{i}$ (see Figure 1 right), (b) is by the conditional Poisson matching lemma on $((\{\check{U}_{1j}\}_{j},M_{2}),\,(U_{2},M_{2}),\,Y_{2},\,P_{U_{2}|Y_{2}}\times P_{M_{2}})$, and (c) is because $\{\check{U}_{1,j+\mathbf{1}\{j\geq K\}}\}_{j\in[1:\mathsf{J}-1]}$ (the $\check{U}_{1j}$’s not selected as $U_{1}$) are independent of $(U_{1},U_{2},Y_{2},M_{2})$, $\mathbf{E}[2^{\iota_{U_{1};U_{2}}(\check{U}_{1,j+\mathbf{1}\{j\geq K\}};U_{2})}\,|\,U_{2}]=1$, and Jensen’s inequality. Hence,

[TABLE]

Therefore there exist fixed realizations of the Poisson processes attaining the desired bound. ∎

IX One-shot Distributed Lossy Source Coding

The one-shot distributed lossy source coding setting is described as follows. Let $(X_{1},X_{2})\sim P_{X_{1},X_{2}}$. Upon observing $X_{j}$, encoder $j$ produces $M_{j}\in[1:\mathsf{L}_{j}]$, $j=1,2$. The decoder observes $M_{1},M_{2}$ and recovers $\hat{Z}_{1}\in\mathcal{Z}_{1}$, $\hat{Z}_{2}\in\mathcal{Z}_{2}$ with probability of excess distortion $P_{e}=\mathbf{P}\{\mathsf{d}_{1}(X_{1},\hat{Z}_{1})>\mathsf{D}_{1}\;\mathrm{or}\;\mathsf{d}_{2}(X_{2},\hat{Z}_{2})>\mathsf{D}_{2}\}$, where $\mathsf{d}_{j}:\mathcal{X}_{j}\times\mathcal{Z}_{j}\to\mathbb{R}_{\geq 0}$ is a distortion measure for $j=1,2$.

We show a one-shot version of the Berger-Tung inner bound [11, 12].

Theorem 7**.**

Fix any $P_{U_{1}|X_{1}}$, $P_{U_{2}|X_{2}}$ and functions $z_{j}:\mathcal{U}_{1}\times\mathcal{U}_{2}\to\mathcal{Z}_{j}$, $j=1,2$. There exists a code for distributed lossy source coding with sources $P_{X_{1}},P_{X_{2}}$ and message sizes $\mathsf{L}_{1},\mathsf{L}_{2}$, with probability of excess distortion

[TABLE]

if all the information density terms are defined, where $(X_{1},X_{2},U_{1},U_{2},Z_{1},Z_{2})\sim P_{X_{1},X_{2}}P_{U_{1}|X_{1}}P_{U_{2}|X_{2}}\delta_{z_{1}(U_{1},U_{2})}\delta_{z_{2}(U_{1},U_{2})}$. As a result, for $\gamma>0$,

[TABLE]

The logarithmic term in (8) (or the last term in (9)) results in an $O(n^{-1}\log n)$ penalty on the rate in the finite blocklength regime, and does not affect the second order result. Ignoring the last term in (9), the error event in (9) is a strict subset of that in [5, eqn (47)]. This is because the error event in [5] is a superset of (9) by Fourier-Motzkin elimination on $\mathsf{J}_{1},\mathsf{J}_{2}$ in the error event in [5], but the reverse is not true since Fourier-Motzkin elimination only guarantees the existence of random variables for $\mathsf{J}_{1},\mathsf{J}_{2}$ (that depend on the information density terms) satisfying the bounds, but $\mathsf{J}_{1},\mathsf{J}_{2}$ must be constants since they are parameters of the code construction in [5].

We now prove the result. Unlike previous approaches, our proof does not require binning. The encoders are the same as those for point-to-point lossy source coding.

Proof:

Let $\{(\bar{U}_{1,i},\bar{M}_{1,i}),T_{1,i}\}_{i\in\mathbb{N}}$, $\{(\bar{U}_{2,i},\bar{M}_{2,i}),T_{2,i}\}_{i\in\mathbb{N}}$ be two independent Poisson processes with intensity measures $P_{U_{1}}\times P_{M_{1}}\times\lambda_{\mathbb{R}_{\geq 0}}$ and $P_{U_{2}}\times P_{M_{2}}\times\lambda_{\mathbb{R}_{\geq 0}}$ respectively, independent of $X_{1},X_{2}$. The encoding functions are $M_{j}=(\tilde{M}_{j})_{P_{U_{j}|X_{j}}(\cdot|X_{j})\times P_{M_{j}}}$, $j=1,2$ (which Poisson process we are referring to can be deduced from whether we are discussing $M_{1}$ or $M_{2}$). Also define $U_{j}=(\tilde{U}_{j})_{P_{U_{j}|X_{j}}(\cdot|X_{j})\times P_{M_{j}}}$, $Z_{j}=z_{j}(U_{1},U_{2})$, $j=1,2$. For the decoding function, let $\check{U}_{1k}=(\tilde{U}_{1})_{P_{U_{1}}\times\delta_{M_{1}}}(k)$ for $k\in\mathbb{N}$, $\hat{U}_{2}=(\tilde{U}_{2})_{\sum_{k=1}^{\infty}\phi(k)P_{U_{2}|U_{1}}(\cdot|\check{U}_{1k})\times\delta_{M_{2}}}$ where $\phi(k)\propto k^{-1}(\log(k+2))^{-2}$ with $\sum_{k=1}^{\infty}\phi(k)=1$, and $\hat{U}_{1}=(\tilde{U}_{1})_{P_{U_{1}|U_{2}}(\cdot|\hat{U}_{2})\times\delta_{M_{1}}}$, $\hat{Z}_{j}=z_{j}(\hat{U}_{1},\hat{U}_{2})$, $j=1,2$. Note that $(M_{1},M_{2},X_{1},X_{2},U_{1},U_{2},Z_{1},Z_{2})\sim P_{M_{1}}\times P_{M_{2}}\times P_{X_{1},X_{2}}P_{U_{1}|X_{1}}P_{U_{2}|X_{2}}\delta_{z_{1}(U_{1},U_{2})}\delta_{z_{2}(U_{1},U_{2})}$.

Let $K=\Upsilon_{P_{U_{1}|X_{1}}(\cdot|X_{1})\times P_{M_{1}}\|P_{U_{1}}\times\delta_{M_{1}}}(1)$ (using the Poisson process $\{(\bar{U}_{1,i},\bar{M}_{1,i}),T_{1,i}\}_{i\in\mathbb{N}}$). By the conditional generalized Poisson matching lemma on $(X_{1},\,1,\,(U_{1},M_{1}),\,M_{1},\,P_{U_{1}}\times\delta_{M_{1}})$ (note that $P_{U_{1},M_{1}|X_{1}}=P_{U_{1}|X_{1}}\times P_{M_{1}}$), almost surely,

[TABLE]

Since $\{\check{U}_{1k}\}_{k}$ is a function of $\{(\bar{U}_{1,i},\bar{M}_{1,i}),T_{1,i}\}_{i}$ and $M_{1}$, we have $\{\check{U}_{1k}\}_{k}\leftrightarrow(X_{1},X_{2},U_{1},U_{2},M_{2})\leftrightarrow\{(\bar{U}_{2,i},\bar{M}_{2,i}),T_{2,i}\}_{i}$. By the conditional Poisson matching lemma on $(X_{2},\,(U_{2},M_{2}),\,(\{\check{U}_{1k}\}_{k},M_{2}),\,\sum_{k=1}^{\infty}\phi(k)P_{U_{2}|U_{1}}(\cdot|\check{U}_{1k})\times\delta_{M_{2}})$ (note that $P_{U_{2},M_{2}|X_{2}}=P_{U_{2}|X_{2}}\times P_{M_{2}}$), almost surely,

[TABLE]

where (a) is by Proposition 6, and (b) is by $K\leftrightarrow(U_{1},X_{1})\leftrightarrow(X_{2},U_{2},M_{2})$, (10) and Jensen’s inequality. By the conditional Poisson matching lemma on $(X_{1},\,(U_{1},M_{1}),\,(U_{2},M_{1}),\,P_{U_{1}|U_{2}}\times\delta_{M_{1}})$ (note that $P_{U_{1},M_{1}|X_{1}}=P_{U_{1}|X_{1}}\times P_{M_{1}}$), and $X_{2}\leftrightarrow(X_{1},U_{1},U_{2},M_{1})\leftrightarrow\{(\bar{U}_{1,i},\bar{M}_{1,i}),T_{1,i}\}_{i}$, almost surely,

[TABLE]

We have

[TABLE]

Therefore there exist fixed values of the Poisson processes attaining the desired bound.

For (9), if the event in (9) does not occur, by Proposition 6 with $\alpha=\gamma-1$, $\tilde{\alpha}=\gamma$, $\beta=\iota_{U_{1};U_{2}}(U_{1};U_{2})-\gamma$,

[TABLE]

∎

Remark 3*.*

The reason for the logarithmic term is that we want to translate a bound on $\mathbf{E}[K]$ (given by the generalized Poisson matching lemma) into a bound on $\mathbf{E}[(\phi(K))^{-1}]$ for some distribution $\phi$ over $\mathbb{N}$. Ideally, we wish $(\phi(k))^{-1}\propto k$, but this is impossible since the harmonic series diverges. Therefore we use a slow converging series $\phi(k)\propto k^{-1}(\log(k+2))^{-2}$ instead, resulting in a logarithmic penalty.

If we use $\mathsf{J}^{-1}\mathbf{1}\{k\leq\mathsf{J}\}$ instead of $\phi(k)$ in the proof, we can obtain the following bound for any $\mathsf{J}\in\mathbb{N}$:

[TABLE]

Compared to Theorem 7, this does not contain the logarithmic term, but requires optimizing over $\mathsf{J}$, and may give a worse second order result.

Another choice is to use $g(k)\propto k^{-1}\mathbf{1}\{k\leq\mathsf{J}\}$ instead of $\phi(k)$. We can obtain the following bound for any $\mathsf{J}\in\mathbb{N}$:

[TABLE]

which gives the same second order result as Theorem 7. Nevertheless, we prefer using $\phi(k)$ which eliminates the need for a parameter $\mathsf{J}$ at the decoder.

X One-shot Coding for Multiple Access Channels

The one-shot coding setting for the multiple access channel is described as follows. Upon observing $M_{j}\sim\mathrm{Unif}[1:\mathsf{L}_{j}]$ ($M_{1},M_{2}$ independent), encoder $j$ produces $X_{j}$, $j=1,2$. The decoder observes the output $Y$ of the channel $P_{Y|X_{1},X_{2}}$ and recovers $(\hat{M}_{1},\hat{M}_{2})$. The error probability is $P_{e}=\mathbf{P}\{(M_{1},M_{2})\neq(\hat{M}_{1},\hat{M}_{2})\}$.

We present a one-shot achievability result for the capacity region in [27, 28, 29]. While this result is slightly weaker than that in [3], we include it to illustrate the use of the generalized Poisson matching lemma in simultaneous decoding. Note that the logarithmic term results in an $O(n^{-1}\log n)$ penalty on the rate in the finite blocklength regime, and does not affect the second order result.

Theorem 8**.**

Fix any $P_{X_{1}},P_{X_{2}}$. There exists a code for the multiple access channel $P_{Y|X_{1},X_{2}}$ for messages $M_{j}\sim\mathrm{Unif}[1:\mathsf{L}_{j}]$, $j=1,2$, with the error probability bounded by

[TABLE]

if $P_{X_{1}X_{2}Y}\ll P_{X_{1}}\times P_{X_{2}}\times P_{Y}$, where $(X_{1},X_{2},Y)\sim P_{X_{1}}P_{X_{2}}P_{Y|X_{1},X_{2}}$. As a result, for $\gamma>0$,

[TABLE]

Proof:

Let $\{(\bar{X}_{1,i},\bar{M}_{1,i}),T_{1,i}\}_{i\in\mathbb{N}}$, $\{(\bar{X}_{2,i},\bar{M}_{2,i}),T_{2,i}\}_{i\in\mathbb{N}}$ be two independent Poisson processes with intensity measures $P_{X_{1}}\times P_{M_{1}}\times\lambda_{\mathbb{R}_{\geq 0}}$ and $P_{X_{2}}\times P_{M_{2}}\times\lambda_{\mathbb{R}_{\geq 0}}$ respectively, independent of $M_{1},M_{2}$. The encoding functions are $X_{1}=(\tilde{X}_{1})_{P_{X_{1}}\times\delta_{M_{1}}}$, $X_{2}=(\tilde{X}_{2})_{P_{X_{2}}\times\delta_{M_{2}}}$ (which Poisson process we are referring to can be deduced from whether we are discussing $X_{1}$ or $X_{2}$). For the decoding function, let $\check{X}_{1k}=(\tilde{X}_{1})_{P_{X_{1}|Y}(\cdot|Y)\times P_{M_{1}}}(k)$ for $k\in\mathbb{N}$, $(\hat{X}_{2},\hat{M}_{2})=(\tilde{X}_{2},\tilde{M}_{2})_{\sum_{k=1}^{\infty}\phi(k)P_{X_{2}|X_{1},Y}(\cdot|\check{X}_{1k},Y)\times P_{M_{2}}}$ where $\phi(k)\propto k^{-1}(\log(k+2))^{-2}$ with $\sum_{k=1}^{\infty}\phi(k)=1$, and $\hat{M}_{1}=(\tilde{M}_{1})_{P_{X_{1}|X_{2},Y}(\cdot|\hat{X}_{2},Y)\times P_{M_{1}}}$.

Let $K=\Upsilon_{P_{X_{1}}\times\delta_{M_{1}}\|P_{X_{1}|Y}(\cdot|Y)\times P_{M_{1}}}(1)$ (using the Poisson process $\{(\bar{X}_{1,i},\bar{M}_{1,i}),T_{1,i}\}_{i\in\mathbb{N}}$). By the conditional generalized Poisson matching lemma on $(M_{1},\,1,\,(X_{1},M_{1}),\,Y,\,P_{X_{1}|Y}\times P_{M_{1}})$ (note that $P_{X_{1},M_{1}|M_{1}}=P_{X_{1}}\times\delta_{M_{1}}$), almost surely,

[TABLE]

Since $\{\check{X}_{1k}\}_{k}$ is a function of $\{(\bar{X}_{1,i},\bar{M}_{1,i}),T_{1,i}\}_{i}$ and $Y$, we have $\{\check{X}_{1k}\}_{k}\leftrightarrow(X_{1},X_{2},Y,M_{2})\leftrightarrow\{(\bar{X}_{2,i},\bar{M}_{2,i}),T_{2,i}\}_{i}$. By the conditional Poisson matching lemma on $(M_{2},\,(X_{2},M_{2}),\,(\{\check{X}_{1k}\}_{k},Y),\,\sum_{k=1}^{\infty}\phi(k)P_{X_{2}|X_{1},Y}(\cdot|\check{X}_{1k},Y)\times P_{M_{2}})$ (note that $P_{X_{2},M_{2}|M_{2}}=P_{X_{2}}\times\delta_{M_{2}}$), almost surely,

[TABLE]

where (a) is by Proposition 6, and (b) is by $K\leftrightarrow(X_{1},Y)\leftrightarrow X_{2}$, (12) and Jensen’s inequality. By the conditional Poisson matching lemma on $(M_{1},\,(X_{1},M_{1}),\,(X_{2},Y),\,P_{X_{1}|X_{2},Y}\times P_{M_{1}})$ (note that $P_{X_{1},M_{1}|M_{1}}=P_{X_{1}}\times\delta_{M_{1}}$), almost surely,

[TABLE]

Therefore there exist fixed values of the Poisson processes attaining the desired bound.

For (11), if the event in (11) does not occur, by Proposition 6 with $\alpha=\gamma-1$, $\tilde{\alpha}=\gamma$, $\beta=\iota_{X_{1};X_{2}|Y}(X_{1};X_{2}|Y)-\gamma$,

[TABLE]

∎

Remark 4*.*

If we use $\mathsf{J}^{-1}\mathbf{1}\{k\leq\mathsf{J}\}$ instead of $\phi(k)$ in the proof, we can obtain the following bound for any $\mathsf{J}\in\mathbb{N}$:

[TABLE]

Compared to Theorem 8, this does not contain the logarithmic term, but requires optimizing over $\mathsf{J}$, and may give a worse second order result.

Another choice is to use $g(k)\propto k^{-1}\mathbf{1}\{k\leq\mathsf{J}\}$ instead of $\phi(k)$. We can obtain the following bound for any $\mathsf{J}\in\mathbb{N}$:

[TABLE]

which gives the same second order result as Theorem 8. Nevertheless, we prefer using $\phi(k)$ which eliminates the need for a parameter $\mathsf{J}$ at the decoder.

XI One-shot Channel Resolvability and Soft Covering

The one-shot channel resolvability setting [30] is described as follows. Fix a channel $P_{Y|X}$ and input distribution $P_{X}$. Upon observing an integer $M\sim\mathrm{Unif}[1:\mathsf{L}]$, the encoder applies a deterministic mapping $g:[1:\mathsf{L}]\to\mathcal{X}$ on $M$ to produce $\hat{X}=g(M)$, which is sent through the channel $P_{Y|X}$ and gives the output $\hat{Y}$. The goal is to minimize the total variation distance between $P_{\hat{Y}}$ and $P_{Y}$ ($Y$-marginal of $P_{X}P_{Y|X}$), i.e., $\epsilon:=\|\mathsf{L}^{-1}\sum_{m=1}^{\mathsf{L}}P_{Y|X}(\cdot|g(m))-P_{Y}(\cdot)\|_{\mathrm{TV}}$.

We show a one-shot channel resolvability result using the the Poisson matching lemma. This result can also be regarded as a one-shot soft covering lemma [31].

Proposition 2**.**

Given channel $P_{Y|X}$ and input distribution $P_{X}$ with $P_{XY}\ll P_{X}\times P_{Y}$. Let $\{\check{X}_{m}\}_{m\in[1:\mathsf{L}]}\stackrel{{\scriptstyle iid}}{{\sim}}P_{X}$, then for any $\mathsf{J}\in\mathbb{N}$,

[TABLE]

As a result, for any $0<\gamma\leq\log\mathsf{L}$,

[TABLE]

Hence there exists a code for channel resolvability satisfying the above bounds.

Proof:

Let $\mathfrak{P}=\{\bar{Y}_{i},T_{i}\}_{i\in\mathbb{N}}$ be the points of a Poisson process with intensity measure $P_{Y}\times\lambda_{\mathbb{R}_{\geq 0}}$. Let $M\sim\mathrm{Unif}[1:\mathsf{L}]$, $\{\check{X}_{m}\}_{m\in[1:\mathsf{L}]}\stackrel{{\scriptstyle iid}}{{\sim}}P_{X}$ ($M\perp\!\!\!\perp\{\check{X}_{j}\}_{j}\perp\!\!\!\perp\mathfrak{P}$), and $X=\check{X}_{M}$. Let $Y=\tilde{Y}_{P_{Y|X}(\cdot|X)}$, and $\hat{Y}_{j}=\tilde{Y}_{P_{Y}}(j)$ for $j\in\mathbb{N}$. We have

[TABLE]

where (a) is by the convexity of the total variation distance, and (b) is because $Y\in\{\hat{Y}_{j}\}_{j\in\mathbb{N}}$ almost surely (note that the summation $\sum_{y\in\{\hat{Y}_{j}\}_{j\in\mathbb{N}}}$ ignores multiplicity of elements in $\{\hat{Y}_{j}\}_{j\in\mathbb{N}}$). For the first term, note that since $Y$ is a function of $(X,\mathfrak{P})$, we have $P_{Y|X,\mathfrak{P}}(y|\check{X}_{m},\mathfrak{P})\in\{0,1\}$, and hence

[TABLE]

We have

[TABLE]

For the second term, by the conditional generalized Poisson matching lemma on $(X,\,1,\,Y,\,\emptyset,\,P_{Y})$,

[TABLE]

Hence,

[TABLE]

For (14), substitute $\mathsf{J}=\lceil\gamma 2^{-\gamma}\mathsf{L}\rceil$,

[TABLE]

where (a) is because $\gamma\leq\log\mathsf{L}$, $2\mathsf{L}2^{-\gamma}>1$ and $(1-(1+\alpha)^{-1})^{\beta}\leq 1-(1+\beta^{-1}\alpha)^{-1}$ for $\alpha\geq 0$, $\beta\geq 1$.

∎

Compare this to Theorem 2 in [32] (weakened by substituting $\delta^{\prime}_{p,W,C}\leq C$): for any $\alpha>0$,

[TABLE]

If we assume $1\leq\alpha\leq\mathsf{L}$ and substitute $\gamma=\log(\mathsf{L}/\alpha)$ in (14), we obtain the following slightly weaker bound (within a logarithmic gap from that in [32]):

[TABLE]

Nevertheless, the bound in [32] does not imply (13), so neither bound is stronger than the other.

The channel resolvability or soft covering bound in Proposition 2 can be applied to prove various secrecy and coordination results, e.g. one-shot coding for wiretap channels [33], one-shot channel synthesis [31], and one-shot distributed source simulation [34]. Hence these results can also be proved using the Poisson matching lemma alone. In the next section, we will prove a one-shot result for wiretap channels.

XII One-shot Coding for Wiretap Channels

The one-shot version of the wiretap channel setting [33] is described as follows. Upon observing $M\sim\mathrm{Unif}[1:\mathsf{L}]$, the encoder produces $X$, which is sent through the broadcast channel $P_{Y,Z|X}$. The legitimate decoder observes $Y$ and recovers $\hat{M}$ with error probability $P_{e}=\mathbf{P}\{M\neq\hat{M}\}$. The eavesdropper observes $Z$. Secrecy is measured by the total variation distance $\epsilon:=\|P_{M,Z}-P_{M}\times P_{Z}\|_{\mathrm{TV}}$.

The following bound is a direct result of the generalized Poisson matching lemma and Proposition 2. It is included for demonstration purposes. See [32, 35, 36] for other one-shot bounds (that are not strictly stronger or weaker than ours).

Proposition 3**.**

Fix any $P_{U,X}$. For any $\nu\geq 0$, $\mathsf{K},\mathsf{J}\in\mathbb{N}$, there exists a code for the wiretap channel $P_{Y,Z|X}$, with message $M\sim\mathrm{Unif}[1:\mathsf{L}]$, with average error probability $P_{e}$ and secrecy measure $\epsilon$ satisfying

[TABLE]

if $P_{UY}\ll P_{U}\times P_{Y}$ and $P_{UZ}\ll P_{U}\times P_{Z}$.

Proof:

Let $\mathfrak{P}=\{(\bar{U}_{i},\bar{M}_{i}),T_{i}\}_{i\in\mathbb{N}}$ be the points of a Poisson process with intensity measure $P_{U}\times P_{M}\times\lambda_{\mathbb{R}_{\geq 0}}$ independent of $M$. Let $K\sim\mathrm{Unif}[1:\mathsf{K}]$ independent of $(M,\mathfrak{P})$. The encoder computes $U=\tilde{U}_{P_{U}\times\delta_{M}}(K)$ and generates $X|U\sim P_{X|U}$. The decoder recovers $\hat{M}=\tilde{M}_{P_{U|Y}(\cdot|Y)\times P_{M}}$. We have $(M,K,U,X,Y,Z)\sim P_{M}\times P_{K}\times P_{U,X}P_{Y,Z|X}$. By the conditional generalized Poisson matching lemma on $(M,\,K,\,(U,M),\,Y,\,P_{U|Y}\times P_{M})$ (note that $P_{U,M|M,K}=P_{U}\times\delta_{M}$),

[TABLE]

For the secrecy measure,

[TABLE]

where (a) is by the convexity of total variation distance, and (b) is by Proposition 2 since $\{\tilde{U}_{P_{U}\times\delta_{m}}(k)\}_{k\in[1:\mathsf{K}]}\stackrel{{\scriptstyle iid}}{{\sim}}P_{U}$ for any $m$. Therefore there exists a fixed set of points for $\mathfrak{P}$ satisfying the desired bound. ∎

XIII Strong Functional Representation Lemma and Noncausal Sampling

The generalized Poisson matching lemma can be applied to give a slight improvement on the constant in the strong functional representation lemma in [1], and hence improves on the variable-length channel simulation result in [37], and the result on minimax remote prediction with a communication constraint in [38]. It also gives an achievability bound on the moments for the noncausal sampling setting in [39].

Proposition 4**.**

Let $\{\bar{U}_{i},T_{i}\}_{i\in\mathbb{N}}$ be the points of a Poisson process with intensity measure $\mu\times\lambda_{\mathbb{R}_{\geq 0}}$ over $\mathcal{U}\times\mathbb{R}_{\geq 0}$, and $P,Q$ be probability measures over $\mathcal{U}$ with $P\ll Q\ll\mu$. For any $j\in\mathbb{N}$, $g:\mathbb{R}_{\geq 0}\to\mathbb{R}$ concave nondecreasing, we have

[TABLE]

i.e., $j(dP/dQ)(U)$ dominates $\Upsilon_{P\|Q}(j)-1$ in the second order. As a result, let

[TABLE]

be the upper concave envelope of $xg^{\prime}(x)$, then

[TABLE]

In particular,

[TABLE]

and for $\gamma\in(0,1)$,

[TABLE]

where $D_{\gamma+1}(P\|Q)=\gamma^{-1}\log\mathbf{E}_{U\sim P}\left[\left((dP/dQ)(U)\right)^{\gamma}\right]$ is the Rényi divergence.

Proof:

For $g:\mathbb{R}_{\geq 0}\to\mathbb{R}$ concave nondecreasing, we have

[TABLE]

where (a) is by Jensen’s inequality, and (b) is by the generalized Poisson matching lemma. For any $\alpha,\beta$ such that $xg^{\prime}(x)\leq\alpha x+\beta$ for $x\geq 0$,

[TABLE]

For $g(x)=\log x$, $xg^{\prime}(x)=\log e$, and hence

[TABLE]

For $g(x)=x^{\gamma}$, $\gamma\in(0,1)$, $xg^{\prime}(x)=\gamma x^{\gamma}$ is concave, and hence

[TABLE]

∎

Consider the setting in the strong functional representation lemma [1]: given $(X,Y)$, we want to find a random variable $Z$ independent of $X$ such that $Y$ is a function of $(X,Z)$, and $H(Y|Z)$ is minimized. Take $Z=\{\bar{Y}_{i},T_{i}\}_{i\in\mathbb{N}}$. Applying Proposition 4 on $P=P_{Y|X}(\cdot|X)$, $Q=P_{Y}$, we obtain

[TABLE]

Using Proposition 4 in [1],

[TABLE]

The constant $3.732$ is smaller than that in [1]:

[TABLE]

XIV Conclusions and Discussion

In this paper, we introduced a simple yet versatile approach to achievability proofs via the Poisson matching lemma. By reducing the uses of sub-codebooks and binning, we improved upon existing one-shot bounds on channels with state information at the encoder, lossy source coding with side information at the decoder, broadcast channels, and distributed lossy source coding. The Poisson matching lemma can replace the packing lemma, covering lemma and soft covering lemma to be the only tool needed to prove a wide range of results in network information theory.

In the proofs, random variables (e.g. the channel input and message in channel coding settings, the source and description in source coding settings, the channel output in channel resolvability) are regarded as points in a Poisson process. The Poisson functional representation is applied to map the Poisson process to give the correct conditional distribution. Viewing every random variable in the operational setting as a Poisson process gives a simple, unified and systematic approach to code constructions.

A possible extension is to generalize the Poisson functional representation to the multivariate case. In the proof of Marton’s inner bound for broadcast channels, we had two independent Poisson processes for $U_{1}$ and $U_{2}$ respectively. We first used the process for $U_{1}$ to obtain a list of values for $U_{1}$, then used the list to index into the process for $U_{2}$. A more symmetric approach where we select $(U_{1},U_{2})$ together (similar to the conventional mutual covering approach) using a multivariate version of the Poisson functional representation may be possible. Similarly, for distributed lossy source coding and the multiple access channel, it may be possible to decode both sources/messages simultaneously. While it can be argued that the gain we obtained in broadcast channels and distributed lossy source coding over conventional approaches comes from the asymmetry of our construction (our bounds are asymmetric unlike previous bounds), a symmetric treatment that does not result in a looser bound may be developed in the future.

XV Acknowledgements

The authors acknowledge support from the NSF grants CNS-1527846, CCF-1618145, the NSF Science & Technology Center grant CCF-0939370 (Science of Information), and the William and Flora Hewlett Foundation supported Center for Long Term Cybersecurity at Berkeley.

-A Proof of Lemmas 1 and 3

We first prove Lemma 3. For notational simplicity, we use $\{X_{i}\}_{i\in\mathbb{N}}\sim\mathfrak{P}(\mu)$ to denote that $\{X_{i}\}_{i\in\mathbb{N}}$ is the set of points of a Poisson process with intensity measure $\mu$ (the ordering of the points is ignored). Let $f(u)=(dP/d\mu)(u)$, $g(u)=(dQ/d\mu)(u)$. Let $\{\bar{U}_{i},T_{i}\}_{i\in\mathbb{N}}\sim\mathfrak{P}(\mu\times\lambda_{\mathbb{R}_{\geq 0}})$. Let $\{\check{U}_{k},\check{T}_{k}\}_{k\in\mathbb{N}}$ be the points $(\bar{U}_{i},T_{i})$ where $f(\bar{U}_{i})=0$. By the mapping theorem [14, 15] on the mapping

[TABLE]

we have $\{\psi(\bar{U}_{i},T_{i})\}_{i\in\mathbb{N}}\sim\mathfrak{P}(\delta_{1}\times P\times\lambda_{\mathbb{R}_{\geq 0}}+\delta_{0}\times\mu_{\{f(u)=0\}}\times\lambda_{\mathbb{R}_{\geq 0}})$ (where $\mu_{\{f(u)=0\}}$ denotes $\mu$ restricted to the set $\{u:\,f(u)=0\}$), and hence $\{\tilde{U}_{P}(k),\tilde{T}_{P}(k)\}_{k\in\mathbb{N}}\sim\mathfrak{P}(P\times\lambda_{\mathbb{R}_{\geq 0}})$ (the points in $\{\psi(\bar{U}_{i},T_{i})\}_{i\in\mathbb{N}}$ with $f(\bar{U}_{i})>0$) is independent of $\{\check{U}_{k},\check{T}_{k}\}_{k\in\mathbb{N}}\sim\mathfrak{P}(\mu_{\{f(u)=0\}}\times\lambda_{\mathbb{R}_{\geq 0}})$ (the points in $\{\psi(\bar{U}_{i},T_{i})\}_{k\in\mathbb{N}}$ with $f(\bar{U}_{i})=0$).

Condition on $\tilde{U}_{P}(j)=u$ and $\tilde{T}_{P}(j)=t$ unless otherwise stated. Assume $f(u)>0$ (which happens almost surely since $\tilde{U}_{P}(j)\sim P$) and $g(u)>0$ (otherwise the inequalities in the lemmas trivially hold). Recall that $\tilde{T}_{P}(1)\leq\tilde{T}_{P}(2)\leq\cdots$ by definition. It is straightforward to check that $\{\tilde{U}_{P}(k),\tilde{T}_{P}(k)\}_{k>j}\sim\mathfrak{P}(P\times\lambda_{[t,\infty)})$ independent of $\{\tilde{U}_{P}(k)\}_{k<j}\stackrel{{\scriptstyle iid}}{{\sim}}P$ independent of $\{\tilde{T}_{P}(k)\}_{k<j}\sim\mathrm{Unif}(t\Delta_{*}^{j-1})$, the uniform distribution over the ordered simplex $t\Delta_{*}^{j-1}=\{s^{j-1}:\,0\leq s_{1}\leq\cdots\leq s_{j-1}\leq t\}$ (i.e., $\{\tilde{U}_{P}(k),\tilde{T}_{P}(k)\}_{k<j}$ has the same distribution as $j-1$ i.i.d. points following $P\times\mathrm{Unif}[0,t]$ sorted in ascending order of the second coordinate). We have

[TABLE]

where

[TABLE]

Due to the aforementioned independence between $\{\check{U}_{k},\check{T}_{k}\}_{k\in\mathbb{N}}$, $\{\tilde{U}_{P}(k),\tilde{T}_{P}(k)\}_{k>j}$ and $\{\tilde{U}_{P}(k),\tilde{T}_{P}(k)\}_{k<j}$, we have $A_{0}{\perp\!\!\!\perp}A_{1}{\perp\!\!\!\perp}B$. For $A_{0}$, since $\{\tilde{U}_{P}(k),\tilde{T}_{P}(k)\}_{k}{\perp\!\!\!\perp}\{\check{U}_{k},\check{T}_{k}\}_{k}$, conditioning on $(\tilde{U}_{P}(j),\tilde{T}_{P}(j))=(u,t)$ does not affect the distribution of $\{\check{U}_{k},\check{T}_{k}\}_{k}$, and hence $A_{0}$ follows the Poisson distribution with rate

[TABLE]

For $A_{1}$, since $\{\tilde{U}_{P}(k),\tilde{T}_{P}(k)\}_{k>j}\sim\mathfrak{P}(P\times\lambda_{[t,\infty)})$, $A_{1}$ follows the Poisson distribution with rate

[TABLE]

Hence $A:=A_{0}+A_{1}$ follows the Poisson distribution with rate

[TABLE]

For $B$, since $\{\tilde{U}_{P}(k),\tilde{T}_{P}(k)\}_{k<j}$ has the same distribution as $j-1$ i.i.d. points following $P\times\mathrm{Unif}[0,t]$ sorted in ascending order of the second coordinate, $B$ follows the binomial distribution with number of trials $j-1$ and success probability

[TABLE]

Conditioned on $\tilde{U}_{P}(j)=u$ (without conditioning on $\tilde{T}_{P}(j)$), we have $\tilde{T}_{P}(j)\sim\mathrm{Erlang}(j,1)$, and $(A,B)|\{\tilde{T}_{P}(j)=t\}\sim\mathrm{Poi}(t\alpha(u))\times\mathrm{Bin}(j-1,\beta(u))$. Hence, conditioned on $\tilde{U}_{P}(j)=u$, the distribution of $\Upsilon_{P\|Q}(j)-1=A+B$ is

[TABLE]

i.e., the sum of a negative binomial random variable and an independent binomial random variable. The mean is

[TABLE]

Also,

[TABLE]

For $j=1$,

[TABLE]

-B Proof of the Conditional Poisson Matching Lemma

The conditional Poisson matching lemma is intuitively obvious. The Poisson matching lemma can be equivalently stated as: for any probability measures $\nu,\xi\ll\mu$, the following holds for $\nu$-almost all $u$:

[TABLE]

where $P_{\{\bar{U}_{i},T_{i}\}_{i}\,|\,\tilde{U}_{\nu}=u}$ is the conditional distribution of the Poisson process given $\tilde{U}_{\nu}=u$. Intuitively, we can consider the Poisson matching lemma to be a statement with 3 parameters $\nu,\xi,u$ (ignore the almost-all condition on $u$ for the moment). Since the statement holds for (almost) any $(\nu,\xi,u)$, it also holds for any random choice of $(\nu,\xi,u)$. In particular, it holds for $(\nu,\xi,u)=(P_{U|X}(\cdot|X),\,Q_{U|Y}(\cdot|Y),\,U)$, where $(X,U,Y)\sim P_{X,U,Y}$, which gives the conditional Poisson matching lemma. Note that the probability in the conditional Poisson matching lemma is conditional on $(X,U,Y)$, where $(X,U,Y)\leftrightarrow(\nu,\xi,u)\leftrightarrow\{\bar{U}_{i},T_{i}\}_{i}$, and hence conditioning on $(X,U,Y)$ has the same effect on $\{\bar{U}_{i},T_{i}\}_{i}$ as conditioning on the parameters $(\nu,\xi,u)$.

We now prove the conditional Poisson matching lemma rigorously. Let $(\Omega,\mathcal{F},P_{\{\bar{U}_{i},T_{i}\}_{i}})$ be the probability space for $\{\bar{U}_{i},T_{i}\}_{i}$, the points of a Poisson process with intensity measure $\mu\times\lambda_{\mathbb{R}_{\geq 0}}$ on $\mathcal{U}\times\mathbb{R}_{\geq 0}$ (let $\mathcal{E}$ be the Borel $\sigma$-algebra of $\mathcal{U}$). The Poisson matching lemma can be equivalently stated as: for any probability measures $\nu,\xi\ll\mu$, and $\kappa:\mathcal{U}\times\mathcal{F}\to[0,1]$ a regular conditional probability distribution (RCPD) of $\{\bar{U}_{i},T_{i}\}_{i}$ conditioned on $\tilde{U}_{\nu}(\{\bar{U}_{i},T_{i}\}_{i})$ (i.e., $\kappa$ is a probability kernel, and $P_{\{\bar{U}_{i},T_{i}\}_{i}}(A\cap\tilde{U}_{\nu}^{-1}(B))=\int_{B}\kappa(u,A)\nu(du)$ for any $A\in\mathcal{F}$, $B\in\mathcal{E}$ , where $\tilde{U}_{\nu}^{-1}(B)$ denotes the preimage of $B$ under $\tilde{U}_{\nu}:\Omega\to\mathcal{U}$, note that $\tilde{U}_{\nu}(\{\bar{U}_{i},T_{i}\}_{i})\sim\nu$), then we have

[TABLE]

for $\nu$-almost all $u$.

Consider the conditional Poisson matching lemma. We have the following for $P_{X,U,Y}$-almost all $(x,u,y)$:

[TABLE]

where (a) holds for $P_{X,U,Y}$-almost all $(x,u,y)$ due to $Y\leftrightarrow(X,U)\leftrightarrow\{\bar{U}_{i},T_{i}\}_{i}$, and (b) is by (17) with $(\nu,\xi,\kappa)\leftarrow(P_{U|X}(\cdot|x),\,Q_{U|Y}(\cdot|y),\,P_{\{\bar{U}_{i},T_{i}\}_{i}|X,U}(\cdot|x,\cdot))$, which holds for $P_{U|X}(\cdot|x)$-almost all $u$, and hence holds for $P_{X,U,Y}$-almost all $(x,u,y)$. We now check that $P_{\{\bar{U}_{i},T_{i}\}_{i}|X,U}(\cdot|x,\cdot)$ satisfies the RCPD condition for $P_{X}$-almost all $x$. Since $X{\perp\!\!\!\perp}\{\bar{U}_{i},T_{i}\}_{i}$, we have $P_{\{\bar{U}_{i},T_{i}\}_{i}}(\cdot)=P_{\{\bar{U}_{i},T_{i}\}_{i}|X}(\cdot|x)$ for $P_{X}$-almost all $x$. Since $U=\tilde{U}_{P_{U|X}(\cdot|X)}(\{\bar{U}_{i},T_{i}\}_{i})$, we have $P_{\{\bar{U}_{i},T_{i}\}_{i}|X,U}(\tilde{U}_{P_{U|X}(\cdot|x)}^{-1}(\{u\})\allowbreak\,|\,x,u)=1$ for $P_{X,U}$-almost all $(x,u)$. Hence the following conditions are satisfied for $P_{X}$-almost all $x$:

[TABLE]

[TABLE]

For any $x$ satisfying (18) and (19), we have the following: for all $A\in\mathcal{F}$, $B\in\mathcal{E}$,

[TABLE]

where (a) is by (18), and (b), (c) are by (19).

-C Proof of Theorem 1

Let $\{\bar{X}_{i},T_{i}\}_{i\in\mathbb{N}}$ be the points of a Poisson process with intensity measure $P_{X}\times\lambda_{\mathbb{R}_{\geq 0}}$ independent of $M$. The encoding function is $m\mapsto\tilde{X}_{P_{X}}(m)$ (i.e., $X=\tilde{X}_{P_{X}}(M)$), and the decoding function is $y\mapsto\Upsilon_{P_{X|Y}(\cdot|y)\|P_{X}}(1)$. We have $(M,X,Y)\sim P_{M}\times P_{X}P_{Y|X}$,

[TABLE]

where (a) is by the definition of $\Upsilon$, (b) is by the conditional generalized Poisson matching lemma on $(\emptyset,M,X,Y,P_{X|Y})$, and (c) is by $M{\perp\!\!\!\perp}(X,Y)$ and Jensen’s inequality. Therefore there exists a fixed $\{\bar{x}_{i},t_{i}\}_{i\in\mathbb{N}}$ attaining the desired bound.

$\blacksquare$

A noteworthy property of this construction is that both the encoder and the decoder do not require knowledge of $\mathsf{L}$. The code can transmit any integer $m\in\mathbb{N}$ with error probability $\mathbf{E}\left[1-(1-\min\{2^{-\iota_{X;Y}(X;Y)},\,1\})^{m}\right]$, assuming unlimited common randomness $\{\bar{X}_{i},T_{i}\}_{i\in\mathbb{N}}$ between the encoder and the decoder.

-D Dispersion of Joint Source-Channel Coding

We show a second order result for joint source-channel coding using Theorem 4 that coincides with the optimal dispersion in [10]. Consider an i.i.d. source sequence $W^{k}$ of length $k$, separable distortion measure $\mathsf{d}(w^{k},\hat{z}^{k})=\frac{1}{k}\sum_{i=1}^{k}\mathsf{d}(w_{i},\hat{z}_{i})$, and $n$ uses of the memoryless channel $P_{Y|X}$. Let $P_{Z|W}$ attain the infimum of the rate-distortion function

[TABLE]

The $\mathsf{D}$-tilted information [40] is defined as

[TABLE]

where $Z\sim P_{Z}$ (the unconditional $Z$-marginal of $P_{W}P_{Z|W}$), and $\nu^{*}=-R^{\prime}(\mathsf{D})$ (the derivative exists if the infimum in $R(\mathsf{D})$ is achieved by a unique $P_{Z|W}$ [40]). We invoke a lemma in [40]:

Lemma 5** ([40], Lemma 2).**

If the following conditions hold:

• •

$\inf\{\tilde{\mathsf{D}}\geq 0:\,R(\tilde{\mathsf{D}})<\infty\}<\mathsf{D}<\inf_{z\in\mathcal{Z}}\mathbf{E}[\mathsf{d}(W,z)]$,

• •

the infimum in $R(\mathsf{D})$ is achieved by a unique $P_{Z|W}$,

• •

there exists a finite set $\tilde{\mathcal{Z}}\subseteq\mathcal{Z}$ such that $\mathbf{E}[\min_{z\in\tilde{\mathcal{Z}}}\mathsf{d}(W,z)]<\infty$, and

• •

$\mathbf{E}_{P_{W}\times P_{Z}}[(\mathsf{d}(W,Z))^{9}]<\infty$* (computed assuming $W,Z$ independent),*

then there exist constants $\alpha,\beta,\gamma,k_{0}>0$ such that for $k\geq k_{0}$,

[TABLE]

where $W^{k}\stackrel{{\scriptstyle iid}}{{\sim}}P_{W}$, and $P_{Z^{k}}=P_{Z}^{\otimes k}$.

We now show a second order result.

Proposition 5**.**

Fix $P_{X}$, $0<\epsilon<1$, $n,k\in\mathbb{N}$. We have $P_{e}=\mathbf{P}\{\mathsf{d}(W^{k},\hat{Z}^{k})>\mathsf{D}\}\leq\epsilon$ if the conditions in Lemma 5 are satisfied, $k\geq k_{0}$, and

[TABLE]

where $C:=I(X;Y)$, $V:=\mathrm{Var}[\iota_{X;Y}(X;Y)]$, $\mathcal{V}(\mathsf{D}):=\mathrm{Var}[\jmath_{W}(W,\mathsf{D})]$, and $\eta>0$ is a constant that depends on $P_{X,Y}$ and the distribution of $\jmath_{W}(W,\mathsf{D})$.

Proof:

We have

[TABLE]

where (a) is by Theorem 4, (b) is by Lemma 5, and (c) is by the Berry-Esseen theorem [20, 21, 22] if we let $\eta-\gamma-1$ be a constant given by the Berry-Esseen theorem. ∎

This coincides with the optimal dispersion in [10]. Although this is not a self-contained proof (it requires the lemma in [40] for the dispersion of lossy source coding), it shows how we can obtain the achievability of the dispersion in joint source-channel coding from a result on the dispersion of lossy source coding with little additional effort, using the Poisson matching lemma. This proof is considerably simpler than that in [10].

-E Properties of $\phi(t)$

Let $\phi:\mathbb{R}_{>0}\to\mathbb{R}_{>0}$, $\phi(t)=ct^{-1}(\log(t+2))^{-2}$, where $c>0$ such that $\sum_{j=1}^{\infty}\phi(j)=1$. Note that $(\phi(t))^{-1}$ is convex. It can be checked numerically that $1\leq c\leq 2$. We prove a useful inequality about $\phi(t)$.

Proposition 6**.**

For any $s>0$, $t\geq 1$, we have

[TABLE]

Moreover, if $st\leq 2^{-\alpha}$, $t-1\leq 2^{\beta}$, and $\tilde{\alpha}\geq\max\{\alpha,0\}$, then

[TABLE]

Proof:

Write $\phi^{-1}(t)$ for the inverse function of $\phi$. Since

[TABLE]

we have

[TABLE]

By the convexity of $(\phi(t))^{-1}$ ,

[TABLE]

If $st\leq 2^{-\alpha}$, $t-1\leq 2^{\beta}$, and $\tilde{\alpha}\geq\max\{\alpha,0\}$,

[TABLE]

where the last inequality follows from considering whether $\beta$ is positive or negative, and the inequality $(x+y)^{2}\leq 2x^{2}+2y^{2}$. ∎

-F *Proof of Theorem 5 for Broadcast Channel with Common

Message*

The parameters $\mathsf{K}_{1},\mathsf{K}_{2}$ correspond to rate splitting. We can split $M_{1}\in[1:\mathsf{L}_{1}]$ into $M_{10}\in[1:\mathsf{K}_{1}]$ and $M_{11}\in[1:\lceil\mathsf{L}_{1}\mathsf{K}_{1}^{-1}\rceil]$, and treat $M_{10}$ as part of $M_{0}$ to be decoded by both decoders. Although $M_{10}$ and $M_{11}$ may not be uniformly distributed, we can apply a random cyclic shift to $M_{1}$ such that $M_{1}\sim\mathrm{Unif}[1:\mathsf{K}_{1}\lceil\mathsf{L}_{1}\mathsf{K}_{1}^{-1}\rceil]$ (and hence $M_{10},M_{11}$ are also uniform), and condition on a fixed shift at the end. Also $M_{2}$ can be split similarly. Therefore we assume $\mathsf{K}_{1}=\mathsf{K}_{2}=1$ without loss of generality.

Let $\mathfrak{P}_{0}=\{(\bar{U}_{0,i},\bar{M}_{00,i}),T_{0,i}\}_{i\in\mathbb{N}}$, $\mathfrak{P}_{1}=\{(\bar{U}_{1,i},\bar{M}_{01,i},\bar{M}_{1,i}),T_{1,i}\}_{i\in\mathbb{N}}$, $\mathfrak{P}_{2}=\{(\bar{U}_{2,i},\bar{M}_{02,i},\bar{M}_{2,i}),T_{2,i}\}_{i\in\mathbb{N}}$ be three independent Poisson processes with intensity measures $P_{U_{0}}\times P_{M_{0}}\times\lambda_{\mathbb{R}_{\geq 0}}$, $P_{U_{1}}\times P_{M_{0}}\times P_{M_{1}}\times\lambda_{\mathbb{R}_{\geq 0}}$ and $P_{U_{2}}\times P_{M_{0}}\times P_{M_{2}}\times\lambda_{\mathbb{R}_{\geq 0}}$ respectively, independent of $M_{0},M_{1},M_{2}$.

The encoder would generate $X$ such that

[TABLE]

where $P_{J}=\mathrm{Unif}[1:\mathsf{J}]$, and $\{\check{U}_{1j}\}_{j\in[1:\mathsf{J}]}\in\mathcal{U}_{1}^{\mathsf{J}}$ is an intermediate list (which can be regarded as a sub-codebook). The term $P_{U_{1}|U_{0}}^{\otimes\mathsf{J}}\delta_{\check{U}_{1J}}$ in (20) means that $\{\check{U}_{1j}\}_{j}$ are conditionally i.i.d. $P_{U_{1}|U_{0}}$ given $U_{0}$, and $U_{1}=\check{U}_{1J}$. To accomplish this, the encoder computes $U_{0}=(\tilde{U}_{0})_{P_{U_{0}}\times\delta_{M_{0}}}$, $\check{U}_{1j}=(\tilde{U}_{1})_{P_{U_{1}|U_{0}}(\cdot|U_{0})\times\delta_{M_{0}}\times\delta_{M_{1}}}(j)$ for $j=1,\ldots,\mathsf{J}$,

$U_{2}=(\tilde{U}_{2})_{\mathsf{J}^{-1}\sum_{j=1}^{\mathsf{J}}P_{U_{2}|U_{0},U_{1}}(\cdot|U_{0},\check{U}_{1j})\times\delta_{M_{0}}\times\delta_{M_{2}}}$ (which Poisson process we are referring to can be deduced from whether we are discussing $U_{0}$, $U_{1}$ or $U_{2}$), $(J,U_{1})|(U_{0},\{\check{U}_{1j}\}_{j},U_{2})\sim P_{J,U_{1}|U_{0},\{\check{U}_{1j}\}_{j},U_{2}}$ (where $P_{J,U_{1}|U_{0},\{\check{U}_{1j}\}_{j},U_{2}}$ is derived from (20)), and outputs $X=x(U_{0},U_{1},U_{2})$. It can be verified that (20) is satisfied.

For the decoding function at the decoder $a\in[1:2]$, let $(\check{U}_{0aj},\check{M}_{0aj})=(\tilde{U}_{0},\tilde{M}_{00})_{P_{U_{0}|Y_{a}}(\cdot|Y_{a})\times P_{M_{0}}}(j)$ for $j\in\mathbb{N}$, $(\hat{U}_{a},\hat{M}_{0a},\hat{M}_{a})=(\tilde{U}_{a},\tilde{M}_{0a},\tilde{M}_{a})_{\sum_{j=1}^{\infty}\phi(j)(P_{U_{a}|U_{0},Y_{a}}(\cdot|\check{U}_{0aj},Y_{a})\times\delta_{\check{M}_{0aj}})\times P_{M_{a}}}$ where $\phi(j)\propto j^{-1}(\log(j+2))^{-2}$ with $\sum_{j=1}^{\infty}\phi(j)=1$.

Let $K_{a}=\Upsilon_{P_{U_{0}}\times\delta_{M_{0}}\|P_{U_{0}|Y_{a}}(\cdot|Y_{a})\times P_{M_{0}}}(1)$ (using the Poisson process $\mathfrak{P}_{0}$). By the conditional generalized Poisson matching lemma on $(M_{0},\,1,\,(U_{0},M_{0}),\,Y_{a},\,P_{U_{0}|Y_{a}}\times P_{M_{0}})$, almost surely,

[TABLE]

By (20), $U_{0}=(\tilde{U}_{0})_{P_{U_{0}}\times\delta_{M_{0}}}$, $\check{U}_{1j}=(\tilde{U}_{1})_{P_{U_{1}|U_{0}}(\cdot|U_{0})\times\delta_{M_{0}}\times\delta_{M_{1}}}(j)$, and $(\check{U}_{01j},\check{M}_{01j})=(\tilde{U}_{0},\tilde{M}_{00})_{P_{U_{0}|Y_{1}}(\cdot|Y_{1})\times P_{M_{0}}}(j)$, we have $(M_{0},M_{1},U_{0},J)\perp\!\!\!\perp\mathfrak{P}_{1}$ and $(\{(\check{U}_{01j},\check{M}_{01j})\}_{j},Y_{1})\leftrightarrow(M_{0},M_{1},U_{0},J,U_{1})\leftrightarrow\mathfrak{P}_{1}$ (see Figure (2) middle). Hence by the conditional generalized Poisson matching lemma on $((M_{0},M_{1},U_{0}),\,J,\,(U_{1},M_{0},M_{1}),\,(\{(\check{U}_{01j},\check{M}_{01j})\}_{j},Y_{1}),\,\allowbreak\sum_{j=1}^{\infty}\phi(j)\allowbreak(P_{U_{1}|U_{0},Y_{1}}(\cdot|\check{U}_{01j},Y_{1})\times\delta_{\check{M}_{01j}})\times P_{M_{1}})$, almost surely,

[TABLE]

where (a) is due to $(U_{2},Y_{2})\leftrightarrow(M_{0},M_{1},U_{0},J,U_{1},Y_{1})\leftrightarrow\mathfrak{P}_{1}$ (see Figure 2 middle), (b) is due to the aforementioned application of the conditional generalized Poisson matching lemma, (c) is by Proposition 6, and (d) is due to (21) and $K_{1}\leftrightarrow(U_{0},Y_{1},M_{0})\leftrightarrow(J,U_{1},M_{1})$ (see Figure 2 middle).

Also, since $(M_{0},M_{2},U_{0},\{\check{U}_{1j}\}_{j})\perp\!\!\!\perp\mathfrak{P}_{2}$ and $(\{(\check{U}_{02j},\check{M}_{02j})\}_{j},Y_{2})\leftrightarrow(M_{0},M_{2},U_{0},\{\check{U}_{1j}\}_{j},U_{2})\leftrightarrow\mathfrak{P}_{2}$ (see Figure 2 right), by the conditional Poisson matching lemma on $((M_{0},M_{2},U_{0},\{\check{U}_{1j}\}_{j}),\,(U_{2},M_{0},M_{2}),\,(\{(\check{U}_{02j},\check{M}_{02j})\}_{j},Y_{2}),\,\allowbreak\sum_{j=1}^{\infty}\phi(j)\allowbreak(P_{U_{2}|U_{0},Y_{2}}\allowbreak(\cdot|\check{U}_{02j},\allowbreak Y_{2})\times\delta_{\check{M}_{02j}})\times P_{M_{2}})$, almost surely,

[TABLE]

where (a) is due to $(U_{1},Y_{1})\leftrightarrow(U_{0},U_{2},Y_{2},M_{0},M_{2})\leftrightarrow\mathfrak{P}_{2}$ (see Figure 2 right), (b) is due to the aforementioned application of the conditional Poisson matching lemma, (c) is by the same arguments as in the proof of Theorem 6, (d) is by Proposition 6, and (e) is due to (21) and $K_{2}\leftrightarrow(U_{0},Y_{2},M_{0})\leftrightarrow(U_{2},M_{2})$ (see Figure 2 right). Hence

[TABLE]

where $A=(\log(\mathsf{L}_{1}^{-1}\mathsf{J}^{-1}2^{\iota_{U_{1};Y_{1}|U_{0}}(U_{1};Y_{1}|U_{0})}+1)+1)^{2}$, B=\bigl{(}\log((\mathsf{L}_{2}\mathsf{J}^{-1}2^{\iota_{U_{1};U_{2}|U_{0}}(U_{1};U_{2}|U_{0})-\iota_{U_{2},Y_{2}|U_{0}}(U_{2};Y_{2}|U_{0})}+\mathsf{L}_{2}(1-\mathsf{J}^{-1})2^{-\iota_{U_{2},Y_{2}|U_{0}}(U_{2};Y_{2}|U_{0})})^{-1}+1)+1\bigr{)}^{2}.

For (6), if the event in (6) does not occur, by Proposition 6,

[TABLE]