Statistical Learning Aided Decoding of BMST of Tail-Biting Convolutional Code

TL;DR

This paper introduces a statistical learning-based decoding algorithm for BMST of tail-biting convolutional codes, improving decoding performance by replacing CRC with a soft check and leveraging empirical divergence functions.

Contribution

It proposes a novel decoding method combining SLVA with a soft check and statistical learning, enhancing BMST-TBCC decoding without CRC.

Findings

BMST-TBCC achieves performance comparable to polar codes.

The proposed decoding method improves error correction with lower latency.

Numerical results validate the effectiveness of the statistical learning approach.

Abstract

This paper is concerned with block Markov superposition transmission (BMST) of tail-biting convolutional code (TBCC). We propose a new decoding algorithm for BMST-TBCC, which integrates a serial list Viterbi algorithm (SLVA) with a soft check instead of conventional cyclic redundancy check (CRC). The basic idea is that, compared with an erroneous candidate codeword, the correct candidate codeword for the first sub-frame has less influence on the output of Viterbi algorithm for the second sub-frame. The threshold is then determined by statistical learning based on the introduced empirical divergence function. The numerical results illustrate that, under the constraint of equivalent decoding delay, the BMST-TBCC has comparable performance with the polar codes. As a result, BMST-TBCCs may find applications in the scenarios of the streaming ultra-reliable and low latency communication (URLLC) data services.

Tables

Table 1. TABLE I: Average list size needed for T A subscript 𝑇 𝐴 T_{A} and T B subscript 𝑇 𝐵 T_{B}

$\mathrm{SNR}$	2.0	2.5	3.0	3.5	4.0
list size for $T_A$	38	30	23	18	14
list size for $T_B$	25	8.2	2.6	1.3	1.1

Equations

$$\mathrm{subFER}⩽\mathrm{FER}⩽L\cdot \mathrm{subFER}.$$

$$\mathrm{fER}=\frac{\text{number of erroneous decoded sub-frames}}{\text{total number of transmitted sub-frames}}.$$

$$\mathrm{fER}⪅\frac{L}{2}\cdot \mathrm{subFER}.$$

$$B(X)=2^{-n+k}(1+X{)}^{n}A(X),$$

$$D(\mathbf{x},\mathbf{y})=\frac{1}{n}{\log }_2\frac{f(\mathbf{y}|\mathbf{x})}{f(\mathbf{y})}$$

$$f(\mathbf{y})=\prod _{i=0}^{n-1}\left(\frac{1}{2}f(y_i|0)+\frac{1}{2}f(y_i|1)\right).$$

$$D(\mathbf{x},\mathbf{y})\approx {\mathbb{E}}_{Y|V}\left[\frac{1}{2}{\log }_2\frac{f(Y|0)}{f(Y)}+\frac{1}{2}{\log }_2\frac{f(Y|1)}{f(Y)}\right],$$

$$M({\hat{\mathbf{v}}}_{\ell }^{(0)})=D({\hat{\mathbf{v}}}_{\ell }^{(0)},{\mathbf{y}}^{(0)})+D({\tilde{\mathbf{v}}}_{\ell },{\mathbf{y}}^{(1)}\odot \varphi ({\hat{\mathbf{v}}}_{\ell }^{(0)}\mathbf{R})),$$

$${\mathbf{z}}^{(0)}={\mathbf{y}}^{(t+1)}\odot \varphi ({\hat{\mathbf{v}}}_{\max }^{(t)}\mathbf{R}).$$

Full text

I Introduction

It has been pointed out by Shannon [1] that the error free transmission is possible with infinite coding length as long as the transmission rate is below the channel capacity. To approach the channel capacity, a number of powerful iteratively decodable channel codes with long block length have been proposed. For example, low-density parity check (LDPC) codes [2] and turbo codes [3] perform within a few hundredths of a decibel from the Shannon limits under iterative belief propagation (BP) decoding algorithm. However, long codes are not suitable for emerging applications that are sensitive to the delay, such as automated driving, smart grids, industrial automation and medical applications. Designing a good code with strict latency constraint is challenging since most constructions developed for long block length do not deliver good codes in the short block length regime. For this reason, more attention has been paid recently on the design of short and medium block length codes (e.g., a thousand or less information bits) [4].

One solution is to construct LDPC codes by progressive edge growth (PEG) algorithm [5], which can deliver better codes than randomly constructed LDPC codes in short block length regime. Polar codes [6], another promising solution for short packet transmission, have been adopted by 5G control channel. Powerful classical short codes with near maximum likelihood (ML) decoding algorithm was also investigated for low latency communication. In [4], the extended Bose-Chaudhuri-Hocquenghem codes were shown to perform near the normal approximation benchmark under ordered statistics decoding [7]. As shown in [8], in the short block length regime, the tail-biting convolutional codes (TBCCs) with wrap-around Viterbi algorithm [9] outperform significantly state-of-the-art iterative coding schemes.

All the aforementioned codes are block codes with short coding length, whereas convolutional codes with limited decoding window can be alternative choices for the streaming services with strict latency constraint, such as real-time online games and video conference. The comparison in [10] [11] between convolutional codes and PEG-LDPC codes showed that convolutional codes outperform LDPC codes for very short delay when bit error rate is used as a performance metric.

A coding scheme called block Markov superposition transmission (BMST) was proposed in [12] to construct iteratively decodable convolutional codes with long constraint length from simple basic codes. The construction of BMST codes is flexible, in the sense that it applies to all code rates of interest in the interval $(0,1)$ [13] and is capable of supporting a wide range of delays but with a small amount of extra implementation complexity [14]. The extrinsic information transfer (EXIT) chart analysis in [15] showed that BMST codes have near-capacity performance in the waterfall region and an error floor that can be controlled by the encoding memory. However, even with the sliding window decoding algorithm, the BMST codes still suffer from a large decoding delay, which renders BMST codes unsuitable for low latency communication. This is because the BP decoding algorithm performs far worse than the optimal decoding algorithm when the layers (sub-blocks) become short.

To solve this issue, the semi-random block oriented convolutional code (SRBO-CC) was proposed in [16] with a Cartesian product of short code as the basic code. The SRBO-CC can be decoded by the sequential decoding, whose memory load is heavy due to the requirement of a large amount of stack memory. In [17], taking the truncated convolutional code as the basic code, we proposed a list decoding algorithm for SRBO-CC. However, the frame error rate of short convolutional codes without termination is relatively high. In this paper, we take a powerful short TBCC as the basic code to build a BMST-TBCC system, where the random interleaver is replaced by a totally random linear transformation. The BMST-TBCC can be decoded by a successive cancellation decoding algorithm, whose performance depends critically on the performance of the first sub-frame. To recover the first sub-frame reliably, list decoding is conducted and the transmitted codeword is identified from the list with the help of the empirical divergence function. Simulation results show that the BMST-TBCC with successive cancellation decoding algorithm is competitive with the polar codes and that the performance-complexity tradeoffs can be achieved by adjusting the statistical threshold.

II BMST of Tail-biting Convolutional Code

II-A Encoding

Let $\boldsymbol{u}=(\boldsymbol{u}^{(0)},\boldsymbol{u}^{(1)},\cdots,\boldsymbol{u}^{(L-1)})$ be the data to be transmitted, where $\boldsymbol{u}^{(t)}=(u^{(t)}_{0},u^{(t)}_{1},\cdots,u^{(t)}_{k-1})\in\mathbb{F}_{2}^{k}$ for $0\leqslant t\leqslant L-1$. The encoding algorithm of BMST-TBCC with basic code $\mathscr{C}$ is described in Algorithm 1 (see Fig. 1 for reference). We see that the main difference from the conventional BMST is the replacement of the random interleaver in the original BMST with a totally random linear transformer $\mathbf{R}$. Also note that we focus on the case with encoding memory one to minimize the rate loss due to the termination.

II-B Performance Metric

Suppose that $\boldsymbol{c}^{(t)}$ is modulated with binary phase-shift keying (BPSK) signals and transmitted over additive white Gaussian noise (AWGN) channels, resulting in a noisy version $\boldsymbol{y}^{(t)}\in\mathbb{R}^{n}$ at the receiver. We focus on a sliding window decoding algorithm, which attempts to recover $\boldsymbol{u}^{(t)}$ from $\boldsymbol{y}^{(t)}$ and $\boldsymbol{y}^{(t+1)}$. In other words, the decoding window is two and hence the decoding delay is $2n$. Given a decoding algorithm, define subFER as the probability that the decoding result $\hat{\boldsymbol{u}}^{(0)}$ is not equal to the transmitted vector $\boldsymbol{u}^{(0)}$ and FER as the probability that the decoding result $\hat{\boldsymbol{u}}$ is not equal to $\boldsymbol{u}$. Clearly, we have

[TABLE]

In practice, we define

[TABLE]

The event that the decoding result $\hat{\boldsymbol{u}}^{(0)}$ is not equal to the transmitted vector $\boldsymbol{u}^{(0)}$ is referred to as the first error event. In general, we say that the first error event occurs at time $t$ if $\hat{\boldsymbol{u}}^{(i)}=\boldsymbol{u}^{(i)}$ for all $i<t$ but $\hat{\boldsymbol{u}}^{(t)}\neq\boldsymbol{u}^{(t)}$. Taking into account that the first error event may cause catastrophic error-propagation, we can prove (omitted here) that

[TABLE]

II-C Weight Enumerating Function

We see that the performance is closely related to the subFER, which, in turn, is closely related to the weight distribution of the truncated code $\mathscr{C}^{(0,1)}=\{(\boldsymbol{c}^{(0)},\boldsymbol{c}^{(1)})|\boldsymbol{c}=(\boldsymbol{c}^{(0)},\cdots,\boldsymbol{c}^{(L)})\;\textrm{is a coded sequence with}\;\boldsymbol{c}^{(0)}\neq\boldsymbol{0}\}$. Let $A(X)$ be the weight enumerating function of the basic code $\mathscr{C}\backslash{\boldsymbol{0}}$ (all non-zero codewords). Then the ensemble weight enumerating function of the truncated code $\mathscr{C}^{(0,1)}$ with $\mathbf{R}$ being totally random is given by

[TABLE]

which can be used to upper-bound subFER if maximum likelihood decoding of $\boldsymbol{c}^{(0)}$ could be implemented based on $(\boldsymbol{y}^{(0)},\boldsymbol{y}^{(1)})$.

III Off-Line Statistical Learning

III-A List Decoding

We assume that the basic code $\mathscr{C}$ can be efficiently decoded by outputting a list of candidate codewords. To avoid messy notation, we assume that a codeword $\boldsymbol{v}\in\mathscr{C}$ is transmitted. Upon receiving its noisy version $\boldsymbol{y}=(y_{0},y_{1},\cdots,y_{n-1})$, the decoder serially outputs a list of candidate codewords $\hat{\boldsymbol{v}}_{\ell}$, $\ell=1,2,\cdots,\ell_{\max}$, where $\ell_{\max}$ is a parameter to trade off the performance against the complexity. We will not focus on the detailed implementation in this paper but simply conduct the serial list Viterbi algorithm (SLVA) [18] with the tail-biting constraint. For ease of notation, we use SLVA($\boldsymbol{y}$, $\ell$) to represent the $\ell$-th output of the SLVA. In particular, SLVA($\boldsymbol{y}$, 1), simply denoted by VA($\boldsymbol{y}$), is the output of the Viterbi algorithm (VA).

For any binary vector $\boldsymbol{x}$, its likelihood is given by $f(\boldsymbol{y}|\boldsymbol{x})=\prod_{i=0}^{n-1}f(y_{i}|x_{i})$, where $f(y_{i}|x_{i})$ is the considered conditional probability density function specified by the modulation and the channel. By the nature of the SLVA, we have $f(\boldsymbol{y}|\hat{\boldsymbol{v}}_{1})\geqslant f(\boldsymbol{y}|\hat{\boldsymbol{v}}_{2})\geqslant\cdots\geqslant f(\boldsymbol{y}|\hat{\boldsymbol{v}}_{\ell_{\max}})$, where $\hat{\boldsymbol{v}}_{\ell}$=SLVA($\boldsymbol{y}$, $\ell$). The list decoding is successful if the transmitted one is included in the list. Obviously, the probability of the list decoding being successful can be close to one by enlarging the list size $\ell_{\max}$. Example 1 shows the performance of TBCC under list decoding.

Example 1

The $16$-state $(2,1,4)$ TBCC defined by the polynomial generator matrix $G(D)=[10111,11001]$ with information length $k=32$ ($n=64$) is considered. The list decoding performance is shown in Fig. 2, where we observe that the performance can be improved by increasing the list size.

For a large list size (e.g., $\ell_{\rm max}=64$), the transmitted codeword is included in the list with high probability. Then the key issue is how to identify the transmitted one from the list. One solution is to invoke the cyclic redundancy check (CRC), as embedded in polar codes [19]. However, the overhead (rate loss) due to the CRC is intolerable especially for short TBCCs. This motivates us to employ the intrinsic memory of the BMST system. The basic observation is that, compared with an erroneous candidate codeword, the correct candidate codeword for the first sub-frame has less influence on the output of Viterbi algorithm for the second sub-frame. To proceed, we need the following concept.

III-B Empirical Divergence Function

For the received vector $\boldsymbol{y}\in\mathbb{R}^{n}$, define an empirical divergence function (EDF) as

[TABLE]

for $\boldsymbol{x}\in\mathbb{F}_{2}^{n}$, where

[TABLE]

Note that, in the above definition, $f(\boldsymbol{y})$ is not equal to $2^{-k}\sum_{\boldsymbol{v}\in\mathscr{C}}f(\boldsymbol{y}|\boldsymbol{v})$ but to $2^{-n}\sum_{\boldsymbol{x}\in\mathbb{F}_{2}^{n}}f(\boldsymbol{y}|\boldsymbol{x})$. Also note that the vector $\boldsymbol{y}$ is not necessarily the noisy version of $\boldsymbol{x}$. We are interested in the following cases.

If $\boldsymbol{v}$ is the transmitted one, we have $D(\boldsymbol{v},\boldsymbol{y})\approx I(X;Y)>0$, where $\approx$ is used to indicate that the EDF is around in probability its expectation for large $n$. Here $I(X;Y)$ is the mutual information between the channel output $Y$ and the uniform binary input $X$. 2. 2.

If $\boldsymbol{x}$ is randomly generated (hence typically not equal to the transmitted one), we have

[TABLE]

which is negative from the concavity of the function $\log_{2}(\cdot)$. 3. 3.

What are the typical values of $D(\hat{\boldsymbol{v}},\boldsymbol{y})$, where $\hat{\boldsymbol{v}}={\rm VA}(\boldsymbol{y})$? Given $\boldsymbol{y}$, since $D(\hat{\boldsymbol{v}},\boldsymbol{y})=\max_{\boldsymbol{v}\in\mathscr{C}}D(\boldsymbol{v},\boldsymbol{y})$, we expect that $D(\hat{\boldsymbol{v}},\boldsymbol{y})\gtrapprox I(X;Y)>0$. 4. 4.

What about $D(\tilde{\boldsymbol{v}},\tilde{\boldsymbol{y}})$? Here $\tilde{\boldsymbol{v}}={\rm VA}(\tilde{\boldsymbol{y}})$ where $\tilde{\boldsymbol{y}}=\boldsymbol{x}\odot\boldsymbol{y}$ with $\boldsymbol{x}$ being a totally random bipolar vector, where $\odot$ stands for component-wise product. That is, we first randomly flip the received vector, and then execute the VA to find the first candidate codeword $\tilde{\boldsymbol{v}}$. We expect that $D(\tilde{\boldsymbol{v}},\tilde{\boldsymbol{y}})$ is located between $D(\boldsymbol{v},\boldsymbol{y})$ of the first case and $D(\boldsymbol{x},\boldsymbol{y})$ of the second case.

Example 2

Consider the TBCC in Example 1 again and set ${\rm SNR}=4~{}{\rm dB}$, at which the mutual information is $I(X;Y)\approx 0.79$. The histogram is shown in Fig. 3, from which we observed that $D(\boldsymbol{v},\boldsymbol{y})$ is likely to be large with $\boldsymbol{v}$ being the transmitted one (or the output of the VA corresponding to $\boldsymbol{y}$). Note that the statistical behavior of $D(\tilde{\boldsymbol{v}},\tilde{\boldsymbol{y}})$ is different from that of $D(\boldsymbol{x},\boldsymbol{y})$, since $\tilde{\boldsymbol{v}}$ is dependent on $\tilde{\boldsymbol{y}}$. The typical values of $D(\tilde{\boldsymbol{v}},\tilde{\boldsymbol{y}})$ are greater than those of $D(\boldsymbol{x},\boldsymbol{y})$ but less than those of $D(\boldsymbol{v},\boldsymbol{y})$.

The statistical behavior of the EDF can be helpful in the decoding process of BMST-TBCC. In the case when the decoding result of the first sub-frame $\hat{\boldsymbol{v}}^{(0)}=\boldsymbol{v}^{(0)}$, $\boldsymbol{y}^{(1)}\odot\phi(\hat{\boldsymbol{v}}^{(0)}\mathbf{R})$ is the Gaussian noisy version of $\boldsymbol{v}^{(1)}$, where $\phi(\hat{\boldsymbol{v}}^{(0)}\mathbf{R})$ is the BPSK signal corresponding to the binary vector $\hat{\boldsymbol{v}}^{(0)}\mathbf{R}$. In contrast, in the case when $\hat{\boldsymbol{v}}^{(0)}\neq\boldsymbol{v}^{(0)}$, $\boldsymbol{y}^{(1)}\odot\phi(\hat{\boldsymbol{v}}^{(0)}\mathbf{R})$ is the randomly flipped Gaussian noisy version of $\boldsymbol{v}^{(1)}$. Since these two cases have different statistical impact on the EDF, we are able to distinguish with high probability whether $\boldsymbol{y}^{(1)}\odot\phi(\hat{\boldsymbol{v}}^{(0)}\mathbf{R})$ is randomly flipped (equivalently, $\hat{\boldsymbol{v}}^{(0)}$ is erroneous) or not.

IV Successive Cancellation Decoding Algorithm

The BMST-TBCC can be decoded by a sliding window algorithm with successive cancellation, and the critical step is how to recover reliably $\boldsymbol{v}^{(0)}$, which is not interfered by any other sub-frames. In this section, we propose a method to estimate $\boldsymbol{v}^{(0)}$ from $\boldsymbol{y}^{(0)}$ and $\boldsymbol{y}^{(1)}$.

Given $\boldsymbol{y}^{(0)}$, the SLVA is implemented to deliver serially a list of candidates $\hat{\boldsymbol{v}}^{(0)}_{\ell},1\leqslant\ell\leqslant\ell_{\rm max}$. For each candidate codeword, we define a soft metric

[TABLE]

where $\tilde{\boldsymbol{v}}_{\ell}$ is the output of the VA with $\boldsymbol{y}^{(1)}\odot\phi(\hat{\boldsymbol{v}}^{(0)}_{\ell}\mathbf{R})$ as the input. The first term in the right hand side of (8) specifies the EDF between the candidate codeword and the received vector $\boldsymbol{y}^{(0)}$, while the second term is the EDF between the flipped vector $\boldsymbol{y}^{(1)}$ and its corresponding VA output $\tilde{\boldsymbol{v}}_{\ell}$. Both of them are likely to be large in the case when the candidate codeword is the transmitted one. Heuristically, we will set a threshold on $M(\hat{\boldsymbol{v}}^{(0)}_{\ell})$ to check the correctness of the candidate codeword, as illustrated in Example 3.

Example 3

The TBCC in Example 1 is taken as the basic code. We set ${\rm SNR}=3~{}{\rm dB}$ and $\ell_{\rm max}=64$. The histogram is shown in Fig. 4. We set a threshold $T$ to distinguish the correct decoding candidate from the erroneous one. The decoding candidate $\hat{\boldsymbol{v}}^{(0)}_{\ell}$ is treated to be correct only if $M(\hat{\boldsymbol{v}}^{(0)}_{\ell})\geqslant T$, where $T$ is usually set large (e.g., $T=1.2$ in this example) to reduce the probability that an erroneous candidate is mistaken as the correct one. The threshold $T$, depending on SNRs and coding parameters, can be learned off-line and stored for use in the decoding algorithm.

The successive cancellation decoding algorithm for BMST-TBCC is outlined as follows. For the first sub-frame, the decoder employs the SLVA to compute the decoding candidates, which will be checked by a statistical threshold, until finding a qualified one. If the list size reaches the maximum $\ell_{\rm max}$ and no decoding candidate is qualified, the decoder delivers $\hat{\boldsymbol{v}}^{(0)}_{\ell}$ with the maximum $M(\hat{\boldsymbol{v}}^{(0)}_{\ell})$ as output. After removing the effect of the first sub-frame, the second sub-frame is then decoded in the same way. This process will be continued until all sub-frames are decoded. The detailed schedule for the decoding algorithm is summarized in Algorithm 2.

V Simulation Results

The $16$-state $(2,1,4)$ TBCC defined by the polynomial generator matrix $G(D)=[10111,11001]$ is taken as the basic code. The total rate is set to $R=0.49$ by terminating the codes properly.

Example 4

We set $k=32$ and $\ell_{\rm max}=64$. A set of thresholds $T_{A}=1.3,1.35,1.4,1.45,1.5$ are chosen for ${\rm SNR}=2.0,2.5,3.0,3.5,4.0$, respectively. The fER is shown in Fig. 5. For comparison, we have also redrawn the performance curves of the polar code [20] with length 128 (the same decoding delay as the BMST-TBCC). We observe that the BMST-TBCC with successive cancellation decoding is competitive with the polar code.

Example 5

Consider the code in Example 4 again. Another set of thresholds $T_{B}=0.95,1.0,1.05,1.1,1.15$ are chosen for ${\rm SNR}=2.0,2.5,3.0,3.5,4.0$, respectively. The fER is shown in Fig. 6, while the average list size needed for decoding a sub-frame is shown in Table I. It can be seen that the complexity (average list size), at the cost of performance loss, can be reduced by tuning down the threshold. For example, at ${\rm SNR}=4~{}{\rm dB}$, the computational complexity (average list size) can be reduced more than $10$ times if a performance degradation (fER deterioration) is tolerated from $10^{-5}$ to $10^{-4}$.

VI Conclusion

In this paper, a new decoding algorithm has been proposed for BMST-TBCC. The decoder outputs serially a list of decoding candidates and identifies the correct one by a statistical threshold, which can be designed by statistical learning and adjusted to make a tradeoff between performance and complexity. Simulation results have been presented to show the performance of the proposed algorithm with different parameters.

Acknowledgment

This work was supported by the NSF of China (No. 61771499) and the Basic Research Project of Guangdong Provincial NSF (No. 2016A030308008 and No. 2016A030313298).