Free Ride on LDPC Coded Transmission

TL;DR

This paper introduces the concept of accessible capacity to enable reliable superposition transmission of extra bits over existing coded links without additional energy or bandwidth, especially using LDPC codes.

Contribution

It defines accessible capacity for practical channels, proposes superposition encoding and decoding methods, and demonstrates reliable transmission of up to 60 extra bits with LDPC codes.

Findings

Up to 60 extra bits transmitted reliably with LDPC codes.

Accessible capacity characterized as channel capacity minus payload mutual information.

Superposition coding enables zero-cost additional data transmission.

Abstract

In this paper, we formulate a new problem to cope with the transmission of extra bits over an existing coded transmission link (referred to as coded payload link) without any cost of extra transmission energy or extra bandwidth. This is possible since a gap to the channel capacity typically exists for a practical code. A new concept, termed as accessible capacity, is introduced to specify the maximum rate at which the superposition transmission of extra bits is reliable and has a negligible effect on the performance of the coded payload link. For a binary-input output-symmetric (BIOS) memoryless channel, the accessible capacity can be characterized as the difference between the channel capacity and the mutual information rate of the coded payload link, which can be numerically evaluated for very short payload codes. For a general payload code, we present a simple lower bound on the accessible capacity, given by the channel capacity minus the coding rate of the payload code. We then focus on the scenarios where low-density parity-check (LDPC) codes are implemented for the payload link. We propose to transmit extra bits by random superposition for encoding, and exhaustive search (with the aid of statistical learning) for decoding. We further propose, by establishing an auxiliary channel (called syndrome channel) induced from "zero-forcing" over the binary field, to transmit extra bits with structured codes such as repetition codes and first-order Reed-Muller (RM) codes. Numerical results show that up to 60 extra bits can be reliably transmitted along with a rate-1/2 LDPC code of length 8064.

Equations

$$P_{Y|X}(y|1)=P_{Y|X}(\pi (y)|0)$$

$$E^{(t)}_{i}=\left\{\begin{array}[]{ll}0,&\textrm{if}~{}\widehat{U}_{i}^{(t)}=U_{i}^{(t)}\\ 1,&\textrm{if}~{}\widehat{U}_{i}^{(t)}\neq U_{i}^{(t)}\end{array}\right.~{}~{}\textrm{for}~{}0\leqslant i\leqslant k-1~{}\textrm{and}~{}0\leqslant t\leqslant L-1.$$

$$\mathrm{WER}=\frac{1}{L}\sum _{t=0}^{L-1}\text{Pr}\{{\hat{\mathbf{U}}}^{(t)}\ne {\mathbf{U}}^{(t)}\}=\frac{1}{L}\text{E}[\sum _{t=0}^{L-1}\underset{0⩽i⩽k-1}{\max }{E}_i^{(t)}]$$

$$\mathrm{BER}=\frac{1}{kL}\sum _{t=0}^{L-1}\sum _{i=0}^{k-1}\text{Pr}\{{\hat{U}}_i^{(t)}\ne U_i^{(t)}\}=\frac{1}{kL}\text{E}[\sum _{t=0}^{L-1}\sum _{i=0}^{k-1}{E}_i^{(t)}],$$

$$\varphi (\mathbf{u},\mathbf{v})=\mathbf{x}=\mathbf{c}+\mathbf{w},$$

$$\hat{\mathbf{v}}={\psi }_1(\mathbf{y}).$$

$$\hat{\mathbf{u}}={\psi }_0(\mathbf{y}⊞\hat{\mathbf{w}}),$$

$$y_{i}\boxplus w_{i}=\left\{\begin{array}[]{ll}y_{i},&w_{i}=0\\ \pi(y_{i}),&w_{i}=1\end{array}\right.~{}~{}\textrm{for}~{}0\leqslant i\leqslant nL-1.$$

$$\psi (\mathbf{y})=({\psi }_0(\mathbf{y}⊞\hat{\mathbf{w}}),{\psi }_1(\mathbf{y})).$$

$$C_a({\mathcal{C}}_0)=\sup \{R_1:R_1\text{is accessable}\}.$$

$${\lambda }_0⩽{\tilde{\lambda }}_0+{\lambda }_1^{(L)},$$

$${\lambda }_0=$$

$$⩽$$

$$=$$

$$C_a=C_{\text{BIOS}}-\frac{1}{n}I({\mathbf{C}}^n;{\tilde{\mathbf{Y}}}^n),$$

$$P_{{\mathbf{Y}}^n|{\mathbf{W}}^n}(\mathbf{y}|\mathbf{w})=2^{-k}\sum _{\mathbf{c}\in {\mathcal{C}}_0}{P}_{{\mathbf{Y}}^n|{\mathbf{X}}^n}(\mathbf{y}|\mathbf{w}+\mathbf{c}).$$

$$C_a=$$

$$=$$

$$I({\mathbf{C}}^n,{\mathbf{W}}^n;{\mathbf{Y}}^n)=I({\mathbf{C}}^n,{\mathbf{W}}^n,{\mathbf{X}}^n;{\mathbf{Y}}^n)=I({\mathbf{X}}^n;{\mathbf{Y}}^n),$$

$$C_a=$$

$$=$$

$$C_a⩾C_{\text{BIOS}}-R_0.$$

$$C_a=C_{\text{BIOS}}-\frac{1}{n}I({\mathbf{C}}^n;{\tilde{\mathbf{Y}}}^n)⩾C_{\text{BIOS}}-\frac{k}{n}=C_{\text{BIOS}}-R_0.$$

$$\mathrm{Pr}\left\{\left|\frac{1}{nL}\log \frac{P_{{\mathbf{Y}}^{nL}|{\mathbf{X}}^{nL}}({\mathbf{Y}}^{nL}|0^{nL})}{P_{{\mathbf{Y}}^{nL}}({\mathbf{Y}}^{nL})}-C_{\text{BIOS}}\right|<\delta \right\}>1-\epsilon .$$

$$C_{\text{BIOS}}=\text{E}\left[\log \frac{P_{Y|X}(Y|0)}{P_Y(Y)}\right],$$

$$\mathbf{G}=\left[\begin{array}{c}{\tilde{\mathbf{G}}}_0\\ {\mathbf{G}}_1\end{array}\right],$$

$$\widetilde{\bm{G}}_{0}=\left[\begin{array}[]{cccc}\bm{G}_{0}&&&\\ &\bm{G}_{0}&&\\ &&\ddots&\\ &&&\bm{G}_{0}\end{array}\right],$$

$$P_{{\mathbf{Y}}^{nL}|{\mathbf{X}}^{nL}}({\mathbf{y}}^{nL}|\varphi (\mathbf{u},\mathbf{v}))=\underset{{\mathbf{x}}^{nL}\in \mathcal{C}}{\max }{P}_{{\mathbf{Y}}^{nL}|{\mathbf{X}}^{nL}}({\mathbf{y}}^{nL}|{\mathbf{x}}^{nL}).$$

$${\mathcal{A}}^{(L)}=\left\{{\mathbf{y}}^{nL}:\left|\frac{1}{nL}\log \frac{P_{{\mathbf{Y}}^{nL}|{\mathbf{X}}^{nL}}({\mathbf{y}}^{nL}|0^{nL})}{P_{{\mathbf{Y}}^{nL}}({\mathbf{y}}^{nL})}-C_{\text{BIOS}}\right|<\delta \right\}.$$

$${\lambda }_1^{(L)}$$

$$<\epsilon +\sum _{{\mathbf{y}}^{nL}\in {\mathcal{A}}^{(L)}}\mathrm{Pr}\{{\mathbf{y}}^{nL}\}\cdot {\lambda }_1^{(L)}{|}_{{\mathbf{y}}^{nL}}\text{for}L>L_0.$$

Full text

I Introduction

Low-density parity-check (LDPC) codes, discovered by Gallager in 1962 [1], are shown in practice to achieve near-capacity performance. As a result, LDPC codes have been widely adopted in modern communication systems, such as the satellite transmission [2], Wi-Fi [3] and enhanced mobile broadband (eMBB) applications in 5G [4]. In many scenarios, in addition to payload data, a few extra bits are required to be transmitted. For example, in Digital Terrestrial Television (DTV) systems, the transmitter identification (TxID) is used to detect, diagnose and classify the operating status of radio transmitters. Generally, it is required to acquire the TxID without decoding of the DTV signals, especially for weak interference sources [5]. In systems with hybrid automatic retransmission request (HARQ), the ACK/NACK has to be transmitted to indicate correct reception or erroneous reception. Similarly, the control signals can be viewed as extra data to be transmitted, as they typically have high reliability requirements.

One simple approach is to encode the extra bits and the payload data separately. This separation is either because the receiver is only interested in the extra bits or because the reliability requirement of the extra bits is higher. The main drawback of such an approach is that it will inevitably lead to an extra consumption of transmission power and bandwidth. An interest question arises. Is it possible to transmit extra bits with neither bandwidth expansion nor transmission energy increase? The answer is theoretically positive since a gap always exists between the coding rate of the practical payload codes and the channel capacity.

For DTV system, digital watermarking techniques were used to embed TxID signals into the 8-symbol vestigial sideband (8-VSB) signals [5]. This approach does not require extra bandwidth but slightly increases the transmission power. In [6], the authors proposed an additional data transmission scheme using TxID sequences. In [7], a transmission scheme was presented to piggyback an additional lonely bit on coded payload data by using different interleavers. In [8, 9], the authors proposed to use different constellations to transmit extra bits. In [10], blind identification of LDPC codes was proposed to avoid parameter transmission in adaptive coding and modulation. For grant-free random access network, the authors in [11] proposed to embed short messages in pilot transmission. To the best of our knowledge, existing schemes either require extra physical resources such as bandwidth and transmission power or can embed only a very small amount of extra bits into the payload transmission.

Recently, we have proposed in [12] a new coding scheme to transmit extra bits along with LDPC coded data, whereby extra bits are randomly encoded and then superimposed on LDPC-coded payload data. The decoding of extra bits is achieved by exhaustive search with the aid of learning the statistics of the syndromes. The simulation results in [12] show that, with a hard decision rule, ten extra bits can be embedded reliably into a rate-1/2 LDPC code with length $8064$. This paper is more than an extension of [12], having the following contributions.

We derive the accessible capacity of the superposition transmission of extra bits as well as a simple lower bound on the accessible capacity. 2. 2.

For random constructions, we propose an analytic expression to estimate the performance of the hard-decision decoding (HDD) and further develop a soft-decision decoding (SDD) to improve the performance. 3. 3.

We derive an auxiliary channel model (termed as syndrome channel) from a linear transformation defined at the receiver with the parity-check matrix of the LDPC code. We then propose a systematic approach to transmission of extra bits by using structured codes.

To verify the basic ideas, we take repetition codes and first-order Reed-Muller (RM) codes as construction examples. Simulation results show that as many as 60 extra bits can be packed into a rate-1/2 LDPC code with length 8064, while having a negligible effect on the reliability of the payload data. The proposed extra data transmission schemes can be used in practical systems to efficiently transmit extra bits.

The rest of this paper is organized as follows. The problem statements and the system model are introduced in Section II. In the presented model, the accessible capacity for the extra transmission is derived, where also presented is a constructive proof for the lower bound on the accessible capacity. The coding schemes for extra transmission based on random superposition are presented in Section III. In Section IV, new coding schemes using structured codes are proposed for extra transmission, where both encoding and decoding are designed for the derived syndrome channel. Finally, Section V summarizes this paper.

II Accessible Capacity

II-A Payload Transmission

We focus on a coded payload link consisting of a binary linear block code and a binary-input output-symmetric (BIOS) memoryless channel [13]. The detailed formulation is presented as follows.

Encoding

Let $\mathbb{F}_{2}$ be the binary field and $\mathscr{C}_{0}[n,k]$ be a binary linear block code with dimension $k$ and length $n$, referred to as the payload code. Let $\bm{u}=(\bm{u}^{(0)},\bm{u}^{(1)},\dots,\bm{u}^{(L-1)})\in\mathbb{F}_{2}^{kL}$ be the payload data to be transmitted, where $\bm{u}^{(t)}\in\mathbb{F}_{2}^{k}$ are data blocks with size $k$. By performing separately $L$ times the encoding algorithm of $\mathscr{C}_{0}[n,k]$, the payload data are transformed to a sequence of codewords $\bm{c}=(\bm{c}^{(0)},\bm{c}^{(1)},\dots,\bm{c}^{(L-1)})\in\mathbb{F}_{2}^{nL}$, where $\bm{c}^{(t)}\in\mathscr{C}_{0}[n,k]$ is the codeword corresponding to the data block $\bm{u}^{(t)}$. 2. 2)

BIOS Memoryless Channel

A BIOS channel is characterized by an input set $\mathbb{F}_{2}$, an output set $\mathcal{Y}$ (discrete or continuous), and a conditional probability mass (or density) function $P_{Y|X}(y|x),x\in\mathbb{F}_{2},y\in\mathcal{Y}$, satisfying the symmetric condition that

[TABLE]

for some mapping $\pi:\mathcal{Y}\rightarrow\mathcal{Y}$ with $\pi^{2}(y)=y$ for all $y\in\mathcal{Y}$. The transmitted sequence $\bm{c}$ is transmitted over a BIOS memoryless channel, resulting in a received sequence $\bm{y}\in\mathcal{Y}^{nL}$. 3. 3)

Decoding

Upon receiving $\bm{y}$, the receiver attempts to estimate the transmitted payload bits $\bm{u}$. Since the channel is memoryless, this can be done separately by performing $L$ times the decoding algorithm of the payload code. For notational convenience, we denote by $\psi_{0}$ the decoder which accepts $\bm{y}$ as input and delivers as output the estimated payload sequence $\widehat{\bm{u}}=(\widehat{\bm{u}}^{(0)},\widehat{\bm{u}}^{(1)},\dots,\widehat{\bm{u}}^{(L-1)})$, where $\widehat{\bm{u}}^{(t)}\in\mathbb{F}_{2}^{k}$ for $t=0,1,\dots,L-1$. 4. 4)

Error Performance Metric

We use the word error rate (WER) and/or bit error rate (BER) to characterize the performance of the (de)coding scheme. We define random variables

[TABLE]

Then the WER and BER are given by

[TABLE]

and

[TABLE]

where $\textbf{E}[\cdot]$ is the mathematical expectation. Usually, WER is taken as the performance metric for short block codes while BER is for long block codes.

II-B Extra Transmission

Let $\bm{v}\in\mathbb{F}_{2}^{k_{1}}$ be $k_{1}$ extra bits to be transmitted. We are seeking a way to transmit $\bm{v}$ along with $\bm{u}$ without any bandwidth expansion or transmission power increase. One way is to construct a totally new encoding function $\phi$ that maps $(\bm{u},\bm{v})$ into $\mathbb{F}_{2}^{nL}$. We use the term “totally new” to suggest that such an encoding function can be irrelevant to $\mathscr{C}_{0}$. This, however, need change the encoder for the payload transmission and might be inconvenient in some applications. In this paper, we focus on implementing the transmission of extra bits by superposition, as shown in Fig. 1 and described as follows.

Encoding

The payload data sequence $\bm{u}\in\mathbb{F}_{2}^{kL}$ is encoded by the payload code $\mathscr{C}_{0}[n,k]$, while the extra bits $\bm{v}\in\mathbb{F}_{2}^{k_{1}}$ are encoded by a binary block code $\mathscr{C}_{1}[nL,k_{1}]$ with dimension $k_{1}$, resulting in a codeword $\bm{w}$ with length $nL$. The coded sequence $\bm{x}$ is given by

[TABLE]

where “$+$” is the component-wise addition over $\mathbb{F}_{2}$. The coding rates of the payload and extra bits are $R_{0}=k/n$ and $R_{1}=k_{1}/(nL)$, respectively. 2. 2)

BIOS Memoryless Channel

The coded sequence $\bm{x}$ is transmitted over a BIOS memoryless channel, resulting in a received sequence $\bm{y}\in\mathcal{Y}^{nL}$. 3. 3)

Decoding

Given $\bm{y}$, the receiver attempts to decode both the messages $\bm{u}$ and $\bm{v}$. This can be achieved naturally by implementing a successive cancellation decoder. First, a decoder $\psi_{1}$ is used to decode $\bm{v}$ from $\bm{y}$, i.e.,

[TABLE]

Then, by removing the effect of extra coded data, $\bm{u}$ is recovered by the decoder $\psi_{0}$ of $\mathscr{C}_{0}$. That is,

[TABLE]

where $\widehat{\bm{w}}$ is the $\mathscr{C}_{2}[nL,k_{1}]$ codeword corresponding to $\widehat{\bm{v}}$ and the operation $\boxplus$, referred to as flipping operation, is defined as

[TABLE]

For integrity, we denote the decoding function by

[TABLE] 4. 4)

Error Performance Metric

For technical reasons concerning the proofs below, we take WER $\lambda_{1}^{(L)}=\mathrm{Pr}\{\widehat{\bm{V}}\neq\bm{V}\}$ of extra bits as performance metric for the superposition transmission.

We refer to the code $\mathscr{C}_{1}$ for the extra transmission as the free-ride code. By “free-ride”, we mean that the superposition transmission of extra bits requires neither extra bandwidth nor extra transmission power. We will further prove that the extra transmission with properly designed free-ride codes can have a negligible effect on the error performance of the payload transmission. Certainly, nothing is free since we need extra computational loads for encoding/decoding the extra bits.

Definition 1**.**

A rate $R_{1}$ is accessible for the extra transmission, if there exists a sequence of coding/decoding functions $(\mathscr{C}_{1},\psi_{1})$ with coding rates no less than $R_{1}$, such that $\lim_{L\rightarrow\infty}\lambda_{1}^{(L)}=0$.

Definition 2**.**

The accessible capacity for the extra transmission is defined as the supremum of all accessible rates, i.e.,

[TABLE]

The notation $C_{a}(\mathscr{C}_{0})$ indicates that the accessible capacity depends on the payload code $\mathscr{C}_{0}$. When no confusion arises in the context, we use $C_{a}$ instead for ease.

Remark*.*

The accessible capacity defined in this paper is similar to but different from that defined in [14]. In [14], the Gaussian interference channel is considered, where the superposition of the signals occurs at the receiver and is over the real field. As a result, the transmission of the secondary user in [14] requires extra transmission power. In contrast, the transmission of extra bits is considered in this paper and the superposition is tailored over the binary field, which requires no extra transmission power.

We will show by examples that $C_{a}$ can be calculated numerically for short block codes $\mathscr{C}_{0}$ and prove that the extra transmission at an accessible rate can have a negligible effect on the payload transmission in the following sense.

Proposition 1**.**

Let $\lambda_{0}$ and $\widetilde{\lambda}_{0}$ be the WERs with respect to the payload transmission for the scenarios with and without extra transmission, respectively. Let $\lambda_{1}^{(L)}$ be the WER with respect to the extra transmission. Then $\lambda_{0}$ can be bounded by

[TABLE]

indicating that the effect of the extra transmission can be negligibly small when the extra transmission is reliable. That is, this upper bound implies that $\lambda_{0}\rightarrow\widetilde{\lambda}_{0}$ as $\lambda_{1}^{(L)}\rightarrow 0$ since $\lambda_{0}\geqslant\widetilde{\lambda}_{0}$ holds obviously.

Proof.

Consider the successive cancellation decoder as described in (6)-(9). Let $\widehat{\bm{W}}$ be the estimated free-ride codeword corresponding to $\widehat{\bm{V}}=\psi_{1}(\bm{Y})$. Then we have

[TABLE]

where $\psi_{0}^{(t)}(\bm{Y}\boxplus\widehat{\bm{W}})$ is the $t$-th sub-block of $\psi_{0}(\bm{Y}\boxplus\widehat{\bm{W}})$, i.e., $\psi_{0}^{(t)}(\bm{Y}\boxplus\widehat{\bm{W}})=\widehat{\bm{U}}^{(t)}$. ∎

For ease of analysis, we assume that the payload data $\{U_{i}^{(t)}:0\leqslant i<k,0\leqslant t<L\}$ are independent and uniformly distributed (i.u.d.) binary random variables. Then the accessible capacity can be calculated from the maximum mutual information over the transmission link $\bm{w}\rightarrow\bm{y}$.

Theorem 1**.**

Let $\mathscr{C}_{0}$ be the payload code with coding rate $R_{0}=k/n$ over the given BIOS memoryless channel with capacity $C_{\textrm{BIOS}}$. The accessible capacity $C_{a}$ for the extra transmission is given by

[TABLE]

where $I(\bm{C}^{n};\widetilde{\bm{Y}}^{n})$ is the mutual information between the transmitted codeword $\bm{C}^{n}\in\mathscr{C}_{0}$ and the received sequence $\widetilde{\bm{Y}}^{n}$ over the given BIOS channel without extra transmission.

Proof.

Referring to Fig. 1, we consider the transmission link from the coded extra data $\bm{w}$ to the received sequence $\bm{y}$, where the coded payloads are treated as channel noise. This transmission link can be viewed as a block-wise stationary memoryless channel with block length $n$, where the input set is $\mathbb{F}_{2}^{n}$, the output set is $\mathcal{Y}^{n}$, and the transition probability mass (density) function is given by

[TABLE]

Then the accessible capacity can be calculated as

[TABLE]

It is not difficult to verify that, for BIOS, the conditional mutual information $I(\bm{C}^{n};\bm{Y}^{n}|\bm{W}^{n})$ does not depend on the distribution $p(\bm{w}^{n})$ and is equal to $I(\bm{C}^{n};\widetilde{\bm{Y}}^{n})$, where $\widetilde{\bm{Y}}^{n}$ denotes the received sequence without extra data superposition. Also notice that

[TABLE]

since $(\bm{C}^{n},\bm{W}^{n})\rightarrow\bm{X}^{n}\rightarrow\bm{Y}^{n}$ forms a Markov chain. So we have

[TABLE]

where the last equality holds because of an important fact that, when $\bm{W}^{n}$ is uniformly distributed over $\mathbb{F}_{2}^{n}$, the coded sequence $\bm{X}^{n}=\bm{C}^{n}+\bm{W}^{n}$ is also uniformly distributed for whatever distribution of $\bm{C}^{n}$. ∎

Example* 1**.*

To illustrate the accessible capacity, we consider simple payload codes over a discrete time additive white Gaussian noise (AWGN) channel with binary phase shift keying (BPSK) signalling. Two codes with rate $1/2$, repetition code $\mathscr{C}[2,1]$ and the extended Hamming code $\mathscr{C}[8,4]$, are used for the simulation, as shown in Fig. 2. We see that the accessible capacity increases as the signal-to-noise ratio (SNR) increases. We can also see that the accessible capacity with respect to the repetition code is higher than that with respect to the extended Hamming code, suggesting that the weaker the payload code is, the higher the accessible capacity is. Furthermore, we see that, in high SNR region, the accessible capacities with respect to both codes are well predicted by the lower bound derived in the next subsection.

II-C A Lower Bound on Accessible Capacity

From Theorem 1, we see that the accessible capacity is related to the mutual information $I(\bm{C}^{n};\widetilde{\bm{Y}}^{n})$, which is usually difficult to evaluate for long codes. For this reason, we provide here a simple lower bound on the accessible capacity.

Theorem 2**.**

Let $\mathscr{C}_{0}$ be a block code with coding rate $R_{0}=k/n$ over the given BIOS memoryless channel with capacity $C_{\textrm{BIOS}}$. The accessible capacity $C_{a}$ is lower bounded by

[TABLE]

Proof.

Since $I(\bm{C}^{n};\widetilde{\bm{Y}}^{n})\leqslant H(\bm{C}^{n})=H(\bm{U}^{k})\leqslant k$, we have, from Theorem 1,

[TABLE]

∎

The above proof is simple but implies that the lower bound can be tight for the payload codes with low WERs, in which case we can prove that $I(\bm{C}^{n};\widetilde{\bm{Y}}^{n})\approx k$ by invoking Fano’s inequality. Next, we present an alternative proof that is constructive and provides more insights on the construction of the free-ride codes. The basic technique for the proof, similar to that employed in [15], is to apply Markov’s inequality to the decoding error probability and then to invoke the weak law of large numbers (WLLN) as stated in the following lemma.

Lemma 1**.**

Suppose that the zero sequence $\bm{0}^{nL}$ is transmitted over the BIOS channel with capacity $C_{\textrm{BIOS}}$. As $L$ goes to infinity, $\frac{1}{nL}\log\frac{P_{\bm{Y}^{nL}|\bm{X}^{nL}}(\bm{Y}^{nL}|\bm{0}^{nL})}{P_{\bm{Y}^{nL}}(\bm{Y}^{nL})}$ converges to $C_{\textrm{BIOS}}$ in probability, where $P_{\bm{Y}^{nL}}(\bm{Y}^{nL})$ corresponds to the input distribution $P_{X}(1)=1/2$. That is, for any given $\delta,\varepsilon>0$, with $L$ sufficiently large, we have

[TABLE]

Proof.

For a BIOS channel, the capacity $C_{\textrm{BIOS}}$ can be calculated by

[TABLE]

showing that this lemma is the application of the WLLN. ∎

Alternative Proof of Theorem 2.

We need prove that any rate $R_{1}<C_{\textrm{BIOS}}-R_{0}$ is accessible. For each $L\geqslant 1$, we construct a totally random linear block code with dimension $k_{1}=\lceil nLR_{1}\rceil$ and length $nL$ as the free-ride code $\mathscr{C}_{1}$. Let $\bm{G}_{0}$ and $\bm{G}_{1}$ be the generator matrices of $\mathscr{C}_{0}[n,k]$ and $\mathscr{C}_{1}[nL,k_{1}]$, respectively. Then the payload data and the extra bits $(\bm{u},\bm{v})$ are encoded by a linear block code $\mathscr{C}[nL,kL+k_{1}]$ with the following generator matrix

[TABLE]

where $\widetilde{\bm{G}}_{0}$ is the generator matrix of the $L$-fold Cartesian product of $\mathscr{C}_{0}[n,k]$

[TABLE]

and $\bm{G}_{1}$ is the generator matrix of the free-ride code with elements being generated independently according to the Bernoulli distribution with success probability $1/2$.

Without loss of generality, we assume that the all-zero codeword $\bm{0}^{nL}$ is transmitted. We also assume that the output space $\mathcal{Y}$ is discrete for notational simplicity. A maximum likelihood (ML) decoding algorithm $\psi_{1}$ is assumed to decode the extra bits, which outputs $\bm{v}$ if, for some $\bm{u}$, the codeword $\phi(\bm{u},\bm{v})$ is the most likely one, i.e.,

[TABLE]

Define $\delta=(C_{\textrm{BIOS}}-R_{0}-R_{1})/2$ and

[TABLE]

From Lemma 1, for any $\varepsilon>0$, there exists an integer $L_{0}$ such that $\mathrm{Pr}\{\mathcal{A}^{(L)}\}>1-\varepsilon$ whenever $L>L_{0}$. Then from the law of total expectation,

[TABLE]

where $\lambda_{1}^{(L)}|_{\bm{y}^{nL}}$ is the WER of extra transmission conditional on the received sequence $\bm{y}^{nL}$.

Given the received vector $\bm{y}^{nL}$, the ML codeword $\bm{X}^{nL}$ is a random vector over the code ensemble due to the randomness of $\bm{G}_{1}$. In particular, the ML codeword $\bm{X}^{nL}=\bm{u}\widetilde{\bm{G}}_{0}+\bm{v}\bm{G}_{1}$ is an i.u.d. binary random variable sequence once the decoding output $\bm{v}$ is erroneous, i.e., $\bm{v}\neq\bm{0}^{k_{1}}$. Hence, by using the same trick as in [15], the conditional WER of extra transmission can be upper bounded by

[TABLE]

Since

[TABLE]

then from (26) and (27), we have $\lambda_{1}^{(L)}\rightarrow 0$ as $L\rightarrow\infty$. ∎

We borrow the term semi-random from [16] to characterize a block linear code that has a generator matrix consisting of both random rows and non-random rows (see (22) for an example). Then we have, as a by-product, the following corollary.

Corollary 1**.**

Let $\mathscr{C}$ be a semi-random block code with coding rate less than the BIOS channel capacity. Asymptotically, the bits corresponding to the random rows can be decoded arbitrarily reliable.

Proof.

The proof is omitted here. ∎

III Random Free-ride Codes

This section serves to present the original idea of this work by reviewing our previous work. As new achievements, we propose an analytical expression for estimating the HDD performance and develop the SDD for improving the performance.

Theoretically, the extra transmission at rates below the accessible capacity, requiring neither bandwidth expansion nor transmission power increase, can have a negligible effect on the payload transmission. However, to recover one sub-block of the payload data, one must receive all the $L$ involved noisy sub-blocks and decode the extra bits first. Apparently, this extra delay is not desirable in practice for large $L$. In the remainder of this paper, we focus on $L=1$ and assume an LDPC code for the payload link. We also assume that the coded sequence $\bm{x}$ is modulated by BPSK signalling and transmitted over an AWGN channel, resulting in a receiving vector

[TABLE]

where $(-1)^{\bm{x}}$ is the transmitted signal sequence with the $i$-th component being $(-1)^{x_{i}}$ and $\bm{z}$ is a noise vector with each component being drawn independently from a normal distribution with mean zero and variance $\sigma^{2}$.

III-A Syndrome Statistics

In our previous work [12], we have presented a coding scheme based on random superposition to pack extra bits into LDPC coded data. Similar to [17], the decoding of the superposition transmission is aided by an off-line statistical learning. The basic idea is to distinguish the case without from that with superposition of extra bits by the statistical behavior of the syndrome. The detailed analysis is shown as follows.

Let $\bm{H}=(h_{ij})_{m\times n}$ be the parity-check matrix of the LDPC code for the payload transmission, where $m=n-k$. For ease of analysis, we assume that the LDPC code is $(\gamma,\rho)$-regular, i.e., $\bm{H}$ has constant column weight $\gamma$ and constant row weight $\rho$. The payload data $\bm{u}$ are encoded by the LDPC code, while the extra bits $\bm{v}$ are encoded by a random linear block code $\mathscr{C}_{1}$ of length $n$. Then the transmitted codeword is given by $\bm{x}=\bm{u}\bm{G}_{0}+\bm{v}\bm{G}_{1}$, where $\bm{G}_{0}$ is the generator matrix of the payload LDPC code and $\bm{G}_{1}$ is the generator matrix of the free-ride code with elements being generated independently according to the Bernoulli distribution with success probability $1/2$. At the receiver, we make hard-decisions on the received signal $\bm{y}$, resulting in $\bm{\widehat{y}}$. Then we define $N(\bm{s})\triangleq W_{H}((\bm{\widehat{y}}+\bm{s})\bm{H}^{\mathrm{T}})$ for each $\bm{s}\in\mathscr{C}_{1}$, where $\bm{H}^{\mathrm{T}}$ denotes the transpose of $\bm{H}$ and $W_{H}(\cdot)$ denotes the Hamming weight function. In words, $N(\bm{s})$ is the number of unsatisfied parity checks if $\widetilde{\bm{c}}=\bm{\widehat{y}}+\bm{s}$ is viewed as a noisy version of the transmitted LDPC codeword. Due to the existence of the channel noise and the considered random-like free-ride code, $N(\bm{s})$ can be treated as a random variable, whose distribution heavily depends on whether the assumed free-ride codeword $\bm{s}$ is the transmitted one or not. To be precise, we distinguish the following two cases.

Case 1. The free-ride codeword $\bm{s}$ is the transmitted one, i.e., $\bm{s}=\bm{w}=\bm{v}\bm{G}_{1}$. In this case, we have

[TABLE]

where $\bm{\widehat{z}}$ is the error pattern corresponding to the hard decision. Hence $\widetilde{\bm{c}}$ can be viewed as the received sequence by passing the LDPC codeword $\bm{c}$ through a binary symmetric channel (BSC) with crossover probability

[TABLE]

where $Q(x)=\int_{x}^{+\infty}\frac{1}{\sqrt{2\pi}}\exp(-\frac{1}{2}t^{2})\mathrm{d}t$. The probability that a parity check is not satisfied can be calculated as

[TABLE]

Then the distribution of $N_{\bm{s}}$ can be approximated as111This approximation would be accurate only when $\gamma=1$, i.e., none of the parity-check equations were “overlapped”.

[TABLE]

Case 2. The free-ride codeword $\bm{s}$ is not the transmitted one, i.e., $\bm{s}\neq\bm{w}$. In this case, we have

[TABLE]

where $\widetilde{\bm{v}}\neq\bm{0}^{k_{1}}$ is a non-zero sequence corresponding to the free-ride codeword $\bm{s}+\bm{w}$. Hence $\widetilde{\bm{c}}$ can be viewed as the received sequence by passing the LDPC codeword $\bm{c}$ through a BSC with crossover probability 1/2, as the components of the generator matrix $\bm{G}_{1}$ are randomly and independently generated. Consequently, each parity check is not satisfied with a probability of 1/2, leading to an approximate distribution of $N(\bm{s})$ as

[TABLE]

The difference between the distributions (33) and (35) are obvious and can be significant especially when $\rho$ is small and $p_{b}=Q(1/\sigma)\ll 1/2$. In particular, the distribution of $N(\bm{s})$ with the transmitted $\bm{s}$ depends on the SNR, while the one with erroneous $\bm{s}$ is irrelevant to the SNR.

To verify these analysis, we provide the following example.

Example* 2**.*

We consider a rate-1/2 (3,6)-regular LDPC code of length 8064 for the payload data transmission as an example. We set $k_{1}=5$. The histograms of $N(\bm{s})$ with correct candidate and erroneous candidates are show in Fig. 3, from which we see that $N(\bm{s})$ is likely to be large if $\bm{s}$ is erroneous. We also see that the simulations match well with the approximate distributions. As expected, the distribution of $N(\bm{w})$ shifts to left as SNR increases while the distribution of $N(\bm{s})$ with $\bm{s}\neq\bm{w}$ remains unchanged. As a result, the difference between the two distributions becomes significant as SNR increases.

III-B Hard-decision Decoding

The above discussion on the syndrome statistics shows a way to transmit the extra bits along with the LDPC coded payload data. Let $\bm{G}_{1}$ be a $k_{1}\times n$ randomly generated matrix with each element drawn independently from a Bernoulli distribution with success probability $1/2$. The extra bits are first encoded by the linear block code $\mathscr{C}_{1}$ with the generator matrix $\bm{G}_{1}$, and then superimposed on the LDPC coded payload data. In principle, these extra bits can be decoded by an exhaustive search algorithm [12], which outputs $\widehat{\bm{v}}$ such that $N(\widehat{\bm{v}}\bm{G}_{1})=\min_{\bm{s}\in\mathscr{C}_{1}}N(\bm{s})$. For completeness, we reproduce the HDD algorithm of the scheme with randomly coded extra bits in Algorithm 1.

Remark*.*

In Step 2 of Algorithm 1, the computational loads can be reduced at the expense of the memory loads. Indeed, the weight $N(\bm{s})$ can be computed by $N(\bm{s})=W_{H}(\bm{\widehat{y}}\bm{H}^{\mathrm{T}}+\bm{s}\bm{H}^{\mathrm{T}})$, where $\bm{s}\bm{H}^{\mathrm{T}}$ can be pre-computed off-line and stored for use since they do not depend on the channel observations.

III-C Performance and Complexity Analysis

By HDD, the WER of the extra bits is given by

[TABLE]

where the distributions of $N(\bm{w})$ and $N(\bm{s})$ can be approximated by (33) and (35), respectively.

According to the central-limit theorem, for a large $n$, $N(\bm{w})$ can be further approximated by a normal distribution with mean $\mu_{0}=\frac{m}{2}[1-(1-2p_{b})^{\rho}]$ and variance $\sigma_{0}^{2}=\frac{m}{4}[1-(1-2p_{b})^{2\rho}]$, and $N(\bm{s})$ can be approximated by a normal distribution with mean $\mu_{1}=\frac{m}{2}$ and variance $\sigma_{1}^{2}=\frac{m}{4}$. By these approximations, $\lambda_{1}$ can be calculated by

[TABLE]

Compared to the conventional payload data transmission, extra computational loads are required for decoding the extra transmission. The number of extra operations is $\mathcal{O}(2^{k_{1}}n)$, which grows exponentially with the number $k_{1}$ of extra bits. Therefore, due to the limitation of computational resources, only a very few extra bits can be transmitted.

III-D Soft-decision Decoding

Like payload codes, free-ride codes can also be decoded by employing soft decisions to improve the performance. To this end, we turn to the log-likelihood ratio (LLR), defined as

[TABLE]

The basic idea is the same as that for HDD. We check each $\bm{s}\in\mathscr{C}_{1}$ to find the right one.

For each $\bm{s}\in\mathscr{C}_{1}$, the LLRs with respect to $\bm{x}+\bm{s}$ can be obtained by flipping,

[TABLE]

Then the $i$-th parity check is satisfied with LLR given by

[TABLE]

For each $\bm{s}\in\mathscr{C}_{1}$, we define

[TABLE]

Intuitively, we can imagine that $\bm{s}=\bm{w}$ will lead to a larger $\Lambda(\bm{s})$ since, in this case, $\bm{x}+\bm{s}=\bm{c}\in\mathscr{C}_{0}$ and the parity-check equations should hold more likely especially for high SNRs. This can also be verified by statistical learning which is omitted here. To summarize, we present the SDD algorithm in Algorithm 2.

Example* 3**.*

The rate-1/2 (3,6)-regular LDPC code $\mathscr{C}[8064,4032]$ in Example 2 is taken as the payload code. As shown in Fig. 4, the WER performances of hard-decision decoding of the extra bits are well predicted by (37). For the proposed scheme with $k_{1}=5$, we observe that, at SNR 0.5 dB, the scheme achieves a WER lower than $10^{-6}$, which can be traded off with the coding rate for the extra transmission. We see that the WER performance is further improved while using SDD, indicating the potentials to carry more extra bits. Then the sum-product algorithm with maximum 50 iterations is employed for decoding the LDPC coded payload data. We present in Fig. 5 the BER of the code $\mathscr{C}[8064,4032]$. It can be seen that the transmission of extra bits has noticeable effects on the payload data in the region of $\textrm{SNR}\leqslant-0.5$ dB but negligible effects in the region of $\textrm{SNR}\geqslant 0$ dB, which is of interest for the coded payload data. This also suggests that the extra transmission is more reliable than the payload transmission, a figure of merit for some applications.

IV Structured Free-ride Codes

The main issue of the random free-ride codes is the decoding complexity, which grows exponentially with the number of extra bits. In this section, we develop a general construction for free-ride codes and then presented as examples the repetition codes and the RM codes.

IV-A Syndrome Channel

As illustrated in Fig. 1 and characterized by (14), the extra transmission link $\bm{w}\rightarrow\bm{y}$ can be viewed as a block-wise memoryless channel, over which the free-ride codeword is corrupted by two types of “noises”. One is the interference from the payload codeword, while the other is the channel noise. On one hand, to recover the payload data by successive cancellation decoding, we need first recover the extra bits. On the other hand, to recover the extra bits, we need null the interference from the payload link. This can be achieved by transforming the hard-decision vector into a syndrome with the parity-check matrix $\bm{H}$ of the payload code.

More formally, upon receiving $\bm{y}$, we make a hard decision, obtaining $\bm{\widehat{y}}$. The channel can be modeled as

[TABLE]

where $\bm{c}$, $\bm{w}$ and $\bm{\widehat{z}}$ are the transmitted payload codeword, the free-ride codeword and the error pattern, respectively, all of which are binary vectors. For HDD, the channel is a BSC with cross probability

[TABLE]

For SDD, the channel is a time-varying BSC with cross probabilities

[TABLE]

By multiplying the parity-check matrix on both sides of (42), we have the syndrome channel,

[TABLE]

where $\bm{w}\bm{H}^{\mathrm{T}}$ with $\bm{w}\in\mathscr{C}_{1}$ is the input and $\bm{\widehat{z}}\bm{H}^{\mathrm{T}}$ is the error pattern, both of which are binary vectors in $\mathbb{F}_{2}^{m}$. This transformation is similar to the zero-forcing equalization commonly used in multiple-input multiple-output (MIMO) detection, which cancels the interference from the payload link but usually introduces memory among the components of $\bm{\widehat{z}}\bm{H}^{\mathrm{T}}$. The syndrome channel can be approximated as a memoryless BSC, either time-invariant or time-varying, as specified in the following. Denote by $(\widehat{\bm{z}}\bm{H}^{\mathrm{T}})_{i}$ the $i$-th component of $\widehat{\bm{z}}\bm{H}^{\mathrm{T}}$. For HDD, the cross probability is calculated as $\mathrm{Pr}\{(\widehat{\bm{Z}}\bm{H}^{\mathrm{T}})_{i}=1\}=p$ as expressed in (III-A), while for SDD, the cross probabilities can be computed from (44) by

[TABLE]

Since the payload codeword has been nulled in the syndrome channel, the extra bits can be recovered from $\bm{\widehat{y}}\bm{H}^{\mathrm{T}}$. Hence, the performance is determined by the code $\mathscr{C}_{1}\bm{H}^{\mathrm{T}}\in\mathbb{F}_{2}^{m}$. Actually, the basic idea is to transmit the extra bits by employing the coset codes. To be precise, the payload code $\mathscr{C}_{0}$ define a coset partition $\mathbb{F}_{2}^{n}/\mathscr{C}_{0}$. Then the construction of free-ride code $\mathscr{C}_{1}$ can be viewed as selecting some representative elements from the cosets. For the free-ride code to be good, its codewords should be selected from distinct cosets. The whole code $\mathscr{C}$ can be viewed as the direct sum of $\mathscr{C}_{0}$ and $\mathscr{C}_{1}$,

[TABLE]

By considering the linear mapping defined by the parity-matrix $\bm{H}$ from $\mathbb{F}_{2}^{n}$ to $\mathbb{F}_{2}^{m}$, we have the following diagram

[TABLE]

where $\mathscr{C}_{1}\bm{H}^{\mathrm{T}}=\{\bm{s}\bm{H}^{\mathrm{T}}:\bm{s}\in\mathscr{C}_{1}\}$ is referred to as the syndrome code and denoted by $\mathscr{C}_{s}$.

Without loss of generality, we assume that the first $m$ columns of $\bm{H}$ are linearly independent. That is, the parity-check matrix can be written as $\bm{H}=[\bm{H}_{1},\bm{H}_{2}]$ with $\bm{H}_{1}$ being invertible. Then we have the following general procedure to construct a free-ride code.

Construct a syndrome code $\mathscr{C}_{s}[m,k_{1}]$. Denote its generator matrix by $\bm{G}_{s}$. 2. 2.

Construct a free-ride code $\mathscr{C}_{1}[n,k_{1}]$ by specifying a generator matrix $\bm{G}_{1}$ such that

[TABLE]

For example, we can choose

[TABLE]

The encoding of the free-ride code can be implemented directly over $\mathbb{F}_{2}^{n}$ by calculating $\bm{w}=\bm{v}\bm{G}_{1}$ or indirectly over $\mathbb{F}_{2}^{m}$ by first calculating $\bm{w}_{s}=\bm{v}\bm{G}_{s}$ and then finding a pre-image of $\bm{w}_{s}$ in $\mathbb{F}_{2}^{n}$ with respect to the linear mapping defined by $\bm{H}$. For example, we can choose

[TABLE]

The decoding of the free-ride code can be reduced to the decoding of the syndrome code over the syndrome channel. In particular, for HDD, the ML criterion is equivalent to the minimum distance criterion

[TABLE]

For SDD, the LLR corresponding to the syndrome channel is given by

[TABLE]

where $\bm{\Lambda}(\bm{x})$ is obtained from the channel observations, defined by (38). Then the extra bits can be estimated according to the ML criterion

[TABLE]

where $\langle\bm{x},\bm{y}\rangle=\sum_{i}x_{i}y_{i}$ is the inner product.

IV-B Repetition Codes as Syndrome Codes

IV-B1 Transmission of One Extra Bit

To begin with, we consider the case that only one extra bit is transmitted, i.e., $v\in\mathbb{F}_{2}$. Simply, we implement the repetition code as the syndrome code, i.e., $\bm{G}_{s}=(\bm{1}^{m})$, the all-one vector with length $m$. Then the extra bit can be decoded by a majority-logic (MLG) decoder. It is expected that the repetition coded extra bit can be better detected than the randomly coded one, since the repetition code has a larger minimum Hamming distance.

Example* 4**.*

We consider the transmission of one extra bit, in which a rate-1/2 (3,6)-regular LDPC code with length 128 is used as the payload code. The WER performance of the extra bit and the payload data are shown in Fig. 6 and Fig. 7, respectively. As predicted, the repetition coded extra bit achieves a lower error rate than the randomly coded one. We also see that, with SDD, the repetition coded extra bit has a negligible effect on the reliability of the payload data.

IV-B2 Transmission of Multiple Extra Bits

Assume that $k_{1}$ extra bits are required to be transmitted along with an LDPC codeword and $m_{1}=m/k_{1}$ is an integer. We use the $k_{1}$-fold Cartesian product of the repetition code as the syndrome code. The generator matrix of $\mathscr{C}_{s}$ is a $k_{1}\times m$ matrix, given by

[TABLE]

By employing the MLG decoding of the syndrome code, the number of extra operations is $\mathcal{O}(n)$, which is irrelevant to the number of extra bits $k_{1}$.

Example* 5**.*

The rate-1/2 (3,6)-regular LDPC code $\mathscr{C}[8064,4032]$ in Example 2 is used for the payload transmission. The WER performance of the repetition coded extra bits is shown in Fig. 8, where the dashed curves correspond to performance estimates

[TABLE]

obtained by assuming the syndrome channel as an ideal BSC($p$) with $p$ given in (III-A). We see that the simulation results with HDD match well with the estimates. We also see that, with SDD, 18 extra bits can be transmitted with a WER about $10^{-4}$ at $\textrm{SNR}=1.5$ dB by the use of repetition codes.

IV-C Reed-Muller Codes as Syndrome Codes

By comparing the results in Example 3 and Example 5, we see that the repetition coded extra bits have a worse error performance than the randomly coded ones. The degradation of the performance is due to the small minimum distance of the Cartesian product of repetition codes. To tackle this issue, we consider the first-order RM code as the syndrome code, which has both a large minimum distance and a low complexity ML decoding algorithm. We use $\mathrm{RM}(1,\eta)$ to denote the first-order RM code with dimension $\eta+1$ and length $2^{\eta}$. Since the minimum distance of $\mathrm{RM}(1,\eta)$ is half of its length, it is expected that the RM coded extra bits can have comparable error performance to the randomly coded ones. In order to pack more extra bits into an LDPC codeword, Cartesian products of RM codes are implemented as syndrome codes. Note that to fit the length of payload code, punctured $\mathrm{RM}(1,\eta)$ codes can be implemented in practice. The ML decoding of the first order $\mathrm{RM}(1,\eta)$ can be implemented by the Fast Fourier Transform (FFT) algorithm [18]. As a result, the number of extra operations can be reduced to $\mathcal{O}(n\log n)$, which is also irrelevant to the number of extra bits $k_{1}$.

Example* 6**.*

The rate-1/2 (3,6)-regular LDPC code $\mathscr{C}[8064,4032]$ in Example 2 is used for the payload transmission. FFT based soft-decision ML decoding algorithm is employed for decoding the extra bits. The WER performance of the RM coded extra bits is shown in Fig. 9. We see that, when $k_{1}=10$, the RM coded extra bits, with a much lower decoding complexity, can achieve a slightly better performance than the randomly coded ones. The BER of the payload data is shown in Fig. 10. We see that as many as 60 extra bits can be transmitted with a negligible effect on the reliability of the payload data.

V Conclusions

In this paper, we investigated the problem of transmitting extra bits by superposition over the coded payload link without any cost of extra transmission energy or extra bandwidth. We derived the accessible capacity of extra transmission and provided a lower bound on the accessible capacity. For LDPC coded payload links, we first presented the random free-ride coding scheme, which however requires exponential extra computational loads for decoding. To lower down the decoding complexity, we then presented structured free-ride coding schemes. Numerical results showed that, based on RM codes, as many as 60 extra bits can be transmitted reliably over a payload link coded with a length-8064 LDPC code.