Deep Learning Assisted Sum-Product Detection Algorithm for Faster-than-Nyquist Signaling

TL;DR

This paper introduces a deep learning aided sum-product detection algorithm for faster-than-Nyquist signaling, significantly improving detection performance by mitigating residual intersymbol interference with a neural network-enhanced factor graph approach.

Contribution

It proposes a novel neural network integrated sum-product detection algorithm that enhances FTN signaling detection by effectively handling residual ISI and enabling turbo equalization.

Findings

Achieves up to 2.5 dB performance gain over conventional methods.

Uses a simplified convolutional neural network requiring minimal training batches.

Demonstrates improved bit error rate performance in simulations.

Abstract

A deep learning assisted sum-product detection algorithm (DL-SPA) for faster-than-Nyquist (FTN) signaling is proposed in this paper. The proposed detection algorithm concatenates a neural network to the variable nodes of the conventional factor graph of the FTN system to help the detector converge to the a posterior probabilities based on the received sequence. More specifically, the neural network performs as a function node in the modified factor graph to deal with the residual intersymbol interference (ISI) that is not modeled by the conventional detector with a limited number of ISI taps. We modify the updating rule in the conventional sum-product algorithm so that the neural network assisted detector can be complemented to a Turbo equalization. Furthermore, a simplified convolutional neural network is employed as the neural network function node to enhance the detector's performance and the neural network needs a small number of batches to be trained. Simulation results have shown that the proposed DL-SPA achieves a performance gain up to 2.5 dB with the same bit error rate compared to the conventional sum-product detection algorithm under the same ISI responses.

Tables

Table 1. TABLE I: Hyper-parameters for the training of DL-SPA.

Optimizer	Root Mean Square Propagation
Learning rate	0.001
Batch size	360
SNR range (dB)	(3, 8)
Batch per SNR	60
$\gamma$	0.95
$m_{max}$	15
$f$	15 $\left(\tau =0.6\right)$, 20 ($\tau =0.5$)
Filter size	2 $\times$ 4

Equations

$$\displaystyle T_{i}(x_{i})=\text{exp}\bigg{[}\frac{1}{\sigma^{2}}\text{Re}\bigg{\{}y_{i}x_{i}^{*}-\frac{{\bf{G}}_{i,i}}{2}|x_{i}|^{2}\bigg{\}}\bigg{]},\vspace{-5mm}\vspace{-5mm}$$

$$\displaystyle I_{i,j}(x_{i},x_{j})=\text{exp}\bigg{[}-\frac{1}{\sigma^{2}}\text{Re}\big{\{}{\bf{G}}_{i,j}x_{i}x_{j}^{*}\big{\}}\bigg{]},\vspace{-5mm}$$

$$Q_i(x_i)=O_i(x_i)T_i(x_i)\prod _{j\ne i}{q}_{i,j}(x_i),\text{vspace}-5mm$$

$$o_i(x_i)=\frac{Q_i(x_i)}{O_i(x_i)},\text{vspace}-5mm$$

$$p_{i,j}(x_i)=\frac{Q_i(x_i)}{q_{i,j}(x_i)},\text{vspace}-5mm$$

$$q_{i,j}(x_i)=\sum _{x_j}{I}_{i,j}(x_i,x_j)p_{j,i}(x_j)\text{vspace}-5mm$$

$$q_{i,j}(x_i)=\sum _{x_j}{I}_{i,j}(x_i,x_j)w_{j,i}{p}_{j,i}(x_j),\text{vspace}-3mm$$

$$u_i(x_i)=\prod _{j\ne i}{q}_{i,j}(x_i).\text{vspace}-3mm$$

$$Q_i(x_i)=O_i(x_i)T_i(x_i)v_i(x_i)\prod _{j\ne i}{q}_{i,j}(x_i).\text{vspace}-3mm$$

$$\Lambda =\sum _{m=1}^{m_{max}}{\gamma }^{m_{max}-m}{\mathcal{F}}_{ce}(R,{\hat{R}}^m),\text{vspace}-4mm$$

Full text

I Introduction

With the growing demand of high speed data transmissions, faster-than-Nyquist (FTN) signaling [1, 2] has recently regained its popularity. Different from the conventional methods of enhancing the data rate, which normally requires more time/bandwidth/spatial resources, FTN signaling enhances the spectral efficiency by intentionally transmitting the symbols faster than the Nyquist rate without increasing the bandwidth consumption. Therefore, FTN signaling has been largely considered in different communication applications, such as satellite communications [3], and 5G or beyond 5G communications [4].

A major drawback of FTN signaling is that the higher symbol rate induces inevitable and severe intersymbol interference (ISI) at the transmitter side, which requires a very complex detector at the receiver side [2]. For example, the number of states in a BCJR detector increases exponentially with the constellation size and number of ISI taps. For a coded FTN system, the Turbo equalization is usually applied at the receiver, where iterations are performed between the BCJR detector and channel decoder. The overall detection/decoding complexity further increases with respect to the number of iterations. Therefore, designing practical reduced-complexity detectors is a major research topic for FTN signaling. Two M-BCJR algorithms for detecting FTN signaling were proposed in [5] based on the Ungerboeck observation model [6], and they show promising error performance for coded FTN systems by applying Turbo equalization. A soft-in soft-out (SISO) detection algorithm was proposed in [7], where the sum-product algorithm is applied to a suitable factor graph (FG) based on the Ungerboeck observation model. The complexity of the algorithm is linear in the number of interferers during each iteration in contrast to the BCJR algorithm. However, reduced-complexity detection algorithms usually undermine the error performance. For instance, there are mainly two different aspects that may contribute to the performance loss for the SISO algorithm in [7]. Firstly, since the number of ISI taps for FTN signaling is infinite in theory [2], if we only consider the most significant ISI taps in detection, the residual ISI taps will degrade the error performance of the system. Secondly, cycles contained in the FG may accumulate the correlation between the messages during the detection iterations and thus affect the error performance. Therefore, to improve the performance of the existed reduced-complexity detection algorithms for FTN signaling, we consider to utilize a neural network to compensate the performance loss.

Recently, deep learning supplemented detection algorithms and decoding algorithms are explored by researchers to further enhance the performance of a communication system. This includes the research of the autoencoders [8, 9] and the neural network optimization schemes which transform the FGs into neural network systems [10, 11]. For the autoencoders, a neural network system with multiple layers is employed and trained to overcome the issues such as multipath interferences and signal distortions. However, since the connection among the multiple layers of the neural network model does not rely on the mathematical models of the channels, the neural network usually needs a large number of training samples, generally more than $2^{K}$ [12], where $K$ is the information sequence length, to converge to a good performance. On the other hand, the "unfolded" neural network detection or decoding algorithms take advantage of the well-developed channel models [10], which lead to specific neural network connections. However, the flexibility of the neural network designs is neglected. Such a neural network can only optimize the performance based on a constant graph, which may not lead to the globally optimized performance.

In this paper, we propose a neural network approach to compensate the performance loss for the sum-product detection algorithm (SPDA) proposed in [7] for detecting coded FTN signals. We modify the FG of the SPDA by connecting an arbitrary neural network to the variable nodes (VNs) via additional function nodes (FNs), where the tunable parameters over the edges of the FG are optimized through training. The inaccuracy of the messages over the edges of the FG are expected to be compensated by the neural network. Furthermore, a new message updating rule is proposed so that the proposed deep learning assisted sum-product detection algorithm (DL-SPA) can be easily implemented in a Turbo equalization fashion without the need of optimization corresponding to any particular channel decoder. The proposed algorithm maintains the flexility of neural network designs while reducing the number of training samples by taking advantage of the modified FG. Simulation results show that the proposed DL-SPA outperforms the SPDA under an FTN system with the same symbol rate, and it shows a better performance compared to the truncated BCJR algorithm [13] with the same number of considered ISI taps.

II Preliminaries

II-A Coded FTN and system model

Without loss of generality, a model of an FTN system is shown in Fig. 1. Let $\bm{b}$ denote the source data with length $K$. In the transmitter, $\bm{b}$ is convolutionally encoded, resulting in a codeword $\bm{c}$. A sequence of $N$ binary phase-shift keying (BPSK) symbols $\mathbf{x}=[x_{1},x_{2},...,x_{N}]^{\text{T}}$ is generated after interleaving the bits in $\bm{c}$. FTN signals are linear modulation signals of the form $s(t)=\sqrt{E_{s}}\sum_{n}{x_{n}}{h(t-n\tau T)}$, where $\tau$ is the time acceleration factor of FTN signaling [1] and $h(t)$ is a $T$-orthogonal root raised cosine pulse with a roll-off factor $\alpha$. Assume that the channel is corrupted by additive white Gaussian noise (AWGN) with a variance of $\sigma^{2}$. The received sequence $\bm{y}$ after a matched filtering and FTN rate sampling is given by $\bm{y}=\mathbf{G}\mathbf{x}+\bm{\eta}$, where $\mathbf{G}$ is a Toeplitz generator matrix constructed by the ISI taps ${g_{i}}=\int_{-\infty}^{\infty}{h\left(t\right){h^{*}}\left({t-i\tau T}\right){\rm{d}}t}$, $|i|\leq L$ and $\bm{\eta}$ has an autocorrelation matrix $\mathbb{E}[\bm{\eta}{\bm{\eta}}^{H}]=\sigma^{2}\mathbf{G}$. Here, $L$ denotes the number of channel responses with significant energy, i.e., $|{{{g_{k}}}\mathord{\left/{\vphantom{{{g_{l}}}{{g_{0}}}}}\right.\kern-1.2pt}{{g_{0}}}}|\geq 0.01,|k|\leq L$. The rest ISI taps with insignificant energy are therefore negligible and then set to zeros for simplicity.

Once the sequence $\bm{y}$ is observed, the receiver performs the Turbo equalization, where the extrinsic information from the detector and decoder is exchanged iteratively via the interleaver $\Pi$ or deinterleaver ${\Pi^{-1}}$ until the maximum iteration number is reached. The sequence $\hat{\bm{b}}$ as the estimate of $\bm{b}$ is generated after the iteration, which is regarded as the output for the receiver.

II-B Sum-product detection algorithm

A sum-product detection algorithm (SPDA) was proposed in [7] based on the Ungerbock model [6]. Given the received sequence $\bm{y}$, the SPDA algorithm factorizes the $a\ posterior$ probabilities (APPs) $P(\bm{x}|\bm{y})$ of the transmitted sequence $\bm{x}$ mainly based on three FNs:

$\bullet$ $O_{i}(x_{i})$ for $i\in\{1,...N\}$: The $a\ priori$ probability that the symbol $x_{i}$ is transmitted.

$\bullet$ $T_{i}(x_{i})$ for $i\in\{1,...N\}$: The symbol likelihood function that the symbol $x_{i}$ is transmitted based on the received symbol $y_{i}$.

$\bullet$ $I_{i,j}(x_{i},x_{j})$ for $i\in\{1,...N\}$ and $j\in\{1,...N\}$ that $i>j$: The FN that conveys the APPs from the interfering nodes.

The functions of $T_{i}(x_{i})$ and $I_{i,j}(x_{i},x_{j})$ are defined as [7]:

[TABLE]

[TABLE]

where $x_{i}^{*}$ refers to the conjugate of the symbol $x_{i}$, $\text{Re}\{\cdot\}$ represents the function that returns the real part of a value, and ${\bf{G}}_{i,j}=g_{i-j}$ is the ${(i-j)}$-th ISI tap. It is derived in [7] that P(\bm{x}|\bm{y})\propto\prod_{i=1}^{N}\bigg{[}O_{i}(x_{i})T_{i}(x_{i})\prod_{j<i}I_{i,j}(x_{i},x_{j})\bigg{]}.

Define $q_{i,j}(x_{i})$ as the message from the FN $I_{i,j}$ to the VN $x_{i}$, $p_{i,j}(x_{i})$ as the message from the VN $x_{i}$ to the FN $I_{i,j}(x_{i},x_{j})$, $o(x_{i})$ as the message from the VN $x_{i}$ to the FN $O_{i}$, and $Q_{i}(x_{i})$ as the product of all messages incoming to the VN $x_{i}$, respectively. Here, $Q_{i}(x_{i})$ indicates the proportional probability of the (approximated) APP $P(x_{i}|\bm{y})$ [7]. Fig. (2) shows the messages to be updated between two VNs and the SPDA can be summarized as the updates of the following messages [7]:

[TABLE]

[TABLE]

[TABLE]

[TABLE]

The messages $\{p_{i,j}\}$ and $\{q_{i,j}\}$ are initialized to the same positive values [7] and the messages are updated iteratively until the maximum number of iterations is reached. The number of the FNs $I_{i,j}(x_{i})$ linked to each VN $x_{i}$ increases linearly with the number of channel taps $L_{E}$ that are considered by the detector, where $L_{E}\leq L$. Once $O_{i}(x_{i})$, $T_{i}(x_{i})$, and $I_{i,j}(x_{i},x_{j})$ have been initialized, the SPDA conveys messages between the VNs and FNs to iteratively update the APPs of the transmitted symbols.

The SPDA computes the APPs by updating the messages from the FG. However, in practice, only $L_{E}$ taps are considered due to the detection complexity. Moreover, the FG contains short cycles which accumulate the correlation between the messages during the iterative update and will affect the algorithm’s performance, especially for a coded FTN system with short codeword length. Therefore, we propose the DL-SPA to enhance the detection’s performance for a coded FTN system.

III Proposed DL-SPA algorithm

In this section, we introduce the proposed deep learning assisted sum-product algorithm (DL-SPA). We propose to add a neural network to the original FG proposed in [7], where the neural network is linked to the VNs of the FG via additional neuron FNs. The neural network is expected to convey the information from the residual ISI responses which are not considered by the original FG, and reduces the correlation accumulated in the cycles of the FG. We show how the messages to be passed to the neural network are modified to make the DL-SPA suitable for Turbo equalization. One feature of our proposed structure is that we can train the DL-SPA without the prior knowledge of the decoder. Therefore, once the neural network is trained, the extrinsic information from the decoder can be passed to the ISI-detector without further tuning the parameters in the neural network. Furthermore, we propose a convolutional neural network (CNN) structure for DL-SPA for the sake of a simple training process. It also suits the detection algorithm due to the special convolutional property.

III-A New FG model and modified message updating rule

Note that conventional neural network assisted detection or decoding algorithms introduce weights to the FG then unfold the message-passing algorithm to a neural network system for training and optimization [10]. In the SPDA, trainable weights can be attached to the messages $p_{i,j}(x_{i})$, so the update rule in (6) is modified into:

[TABLE]

where $w_{j,i}$ is the weight attached to the message $p_{j,i}(x_{j})$. Training with the additional weights improves the message passing algorithm’s performance in a high SNR region [10]. However, since the same model or connection is employed in the FG and the conventional neural network, the performance improvement by attaching trainable weights to the neural network is limited. In this work, we propose to concatenate a neural network FN $\Phi(x_{1},...,x_{N})$ to the VNs in the FG to compensate the effects of the residual ISI responses and the correlation induced along the short cycles. As shown in Fig. 3, different from the traditional FG, we nest a neural network to the VNs $x_{i}$ of the FG, for $i\in\{1,...,N\}$. There are mainly two aims of nesting a neural network to the FG:

$\bullet$ The neural network connects to all the VNs. It is expected that all ISIs among the VNs are considered by the neural network. This is simpler compared to the FG that considers all ISI taps on one VN.

$\bullet$ The correlation induced during the iteration is expected to be compensated by the neural network. The APPs computation for all the VNs in each iteration can be optimized after tuning the parameters in the neural network.

Define $u_{i}(x_{i})$ as the message from the variable node $x_{i}$ to the FN $\Phi(x_{1},...,x_{N})$ and $v_{i}(x_{i})$ as the message from the FN $\Phi(x_{1},...,x_{N})$ to the variable node $x_{i}$. The conventional sum-product algorithm sums all the intrinsic information for each variable node before passing the extrinsic information to the FN for further processing. This indicates that in a conventional sum-product algorithm, $u_{i}(x_{i})=O_{i}(x_{i})T_{i}(x_{i})\prod_{j\neq i}{q_{i,j}(x_{i})}$. However, in a Turbo equalization, the extrinsic information from the decoder will pass to the detector. Optimizing a Turbo equalization with a neural network system leads to two major problems. Firstly, the training complexity will be largely increased if the decoder is also covered by the neural network layers. Secondly, the optimization of the neural network needs to consider the specific channel decoder, which is inflexible from the design perspective, and undermines the generality of the ISI detector.

To train an ISI detector which is applicable to a Turbo equalization without the prior knowledge of the decoder, we propose to only pass the soft information from the FN $I_{i,j}(x_{i},x_{j})$ to the neural network. This indicates that the message $u_{i}(x_{i})$ will be updated by:

[TABLE]

The final a posterior probability $Q_{i}(x_{i})$ update rule becomes:

[TABLE]

Compared with the APP update rule in Eq. (3), Eq. (9) contains the message $v_{i}(x_{i})$ from the neural network to the VN $x_{i}$.

Since in each iteration of the Turbo equalization, the value of $T_{i}(x_{i})$ is constant, the messages from $T_{i}(x_{i})$ will not be passed to the neural network for further processing. The messages conveyed from the FN $I_{i,j}(x_{i},x_{j})$ contain correlations due to the short cycles in the FG. Besides reducing the training complexity by passing only the messages from the FN $I_{i,j}(x_{i},x_{j})$, the APPs accumulated at each iteration of the detection algorithm are tuned by the neural network’s output messages $v_{i}(x_{i})$. The DL-SPA can be summarized as follows:

Update all the $a\ posterior$ probabilities $\{Q_{i}\}$ as in (9); 2. 2.

Update the messages $\{p_{i,j}\}$ as in (5); 3. 3.

Update the messages $\{q_{i,j}\}$ as in (6); 4. 4.

Update the messages $\{u_{i}\}$ as in (8); 5. 5.

Compute the messages $\{v_{i}\}$ based on the trained $\Phi(x_{1},...,x_{n})$; 6. 6.

If the maximum number of iterations is not reached, then go back to step 1; 7. 7.

Update all the $a\ posterior$ probabilities $\{Q_{i}\}$ as in (9).

III-B DL-SPA with simplified convolutional neural network and its training procedure

We propose to employ a simplified convolutional neural network (CNN) assisted SPDA by considering the special convolution structure of CNN [14]. CNNs are widely used in image recognition systems. Traditional CNNs usually involve several convolutional layers (Conv) and max-pooling layers. The convolutional layer performs the convolution operation of the filters. The filters convolve and stride over the input. The max-pooling layer performs downsampling to reduce the spatial size of the convolved features. A dense layer is appended after the max-pooling layer to provide possibly nonlinear function [14]. In [15], a pure CNN based detection algorithm was proposed, where both max-pooling layers and dense layer are removed to reduce the training complexity, but a large number of convolutional layers and filters are kept. In this paper, we remove the max-pooling layer and simplify the convolutional neural network to only have one convolutional layer. The message $u_{i}(x_{i})$ contains two values of probabilities due to the BPSK modulation. The convolutional layer has $f$ filters and each filter has a size of $2\times\kappa$, where 2 refers to the size of constellations and $\kappa$ indicates that $\kappa$ adjacent messages of $u_{i}(x_{i})$ are considered by the filter. The stride of the filter in the convolutional layer is set to be 1. The filter convolves with the VNs from $u_{1}(x_{1})$ to $u_{N}(x_{N})$. This indicates that all the VNs are processed by the CNN FN. The output of the convolutional layer is reshaped to the same dimension as the input, then sent to a dense layer. The dense layer processes all the filters’ results then output the message $v_{i}(x_{i})$ for $i\in\{1,...N\}$. A rectified linear activation function (ReLU) is used for both the convolutional layer and the dense layer [16]. The CNN has initial weights and biases randomly generated. Every iteration, messages $u_{i}(x_{i})$ will be updated by Eq. (8) then passed to the CNN. Messages are sent back from the CNN to join the APPs accumulation according to Eq. (9). By “unfolding” the iterative message-passing algorithm to a NN system with multiple layers, the overall detection performance can be trained and optimized.

Equation (9) computes the APP of the transmitted symbol $x_{i}$. A sequence of log-likelihood ratios (LLRs) can further be acquired for every iteration of DL-SPA, where the LLR($x_{i}$)=$\text{log}\frac{P(x_{i}=+1|\bm{y})}{P(x_{i}=-1|\bm{y})}$. This allows us to setup a multi-loss function as introduced in [10] and [17] to train the tunable parameters in the FG, which include the weights in Eq. (7) and the weights and biases introduced in the CNN. Define $\hat{R}_{i}^{m}(x_{i})$ as the LLR of the VN $x_{i}$ at the $m$-th iteration and $m_{max}$ as the maximum number of iterations of DL-SPA. Let the cross-entropy function be \mathcal{F}_{ce}(R,\hat{R}^{m})=-\frac{1}{N}\sum_{i=1}^{N}\big{(}R_{x_{i}}\text{log}(\frac{1}{1+e^{-\hat{R}^{m}_{x_{i}}}})+(1-R_{x_{i}})\text{log}(1-\frac{1}{1+e^{-\hat{R}^{m}_{x_{i}}}})\big{)}, where $R$ is the ground-truth label of the transmitted bits. The final loss function for training the neural network is given by:

[TABLE]

where $\gamma<1$ is a discount factor to adjust the loss at each iteration. During the training phase, a batch of random transmitted symbols over a range of signal-to-noise ratios (SNR) is generated as samples to train the neural network. Once the neural network is fully trained, the DL-SPA can be attached to the Turbo equalization to perform as an ISI detector to exchange the extrinsic information with the decoder.

IV Numerical Results

In this section, we evaluate the performance of the proposed DL-SPA scheme over convolutional coded FTN systems. Without loss of generality, we consider coded FTN systems with $\tau=0.5$ and $\tau=0.6$, where the channel code is the terminated (7, 5) 4-state rate-1/2 non-recursive convolutional code (CC). The length of the data bits in both cases is $K=62$ and a BCJR decoder is employed to decode the CC. To get a fair comparison, we also perform the SPDA [7] and truncated-BCJR detection algorithm [13] with terminated ISI trellis. Note that, additional symbols need to be transmitted to terminate the ISI trellis, so that the overall spectral efficiency for the truncated-BCJR algorithm is slightly lower than that of the SPDA and DL-SPA. The hyper-parameters to train the neural network system are shown in Table I:

The initial values of the weights and biases in each iteration’s CNN are randomly generated from a truncated normal distribution with a standard deviation of 0.03.

The bit error rate (BER) of various detection algorithms for the FTN systems is shown in Figs. 5 and 6. Here, DL-SPA($\rho_{max}$, $L_{E}$) indicates the proposed DL-SPA detection method, and BCJR($\rho_{max}$, $L_{E}$) refers to the corresponding truncated BCJR detection algorithm [13]. Here, $\rho_{max}$ indicates the number of iterations of the Turbo equalization. Both DL-SPA and SPDA utilize 15 iterations for updating the messages. It can be seen that for an FTN system with $\tau=0.6$, the SPDA has the worst error performance at a high SNR region, due to the lack of considerations of the residual ISI responses. On the other hand, by adding the neural network FN, the DL-SPA(5, 2) can achieve similar error performance as the SPDA(5, 4). In particular, the DL-SPA(5, 2) shows 2.5 dB and 0.5 dB gain to the SPDA(5, 2) and the BCJR(5, 2), respectively. The DL-SPA(5, 2) is 0.9 dB away from the CC decoding performance without ISI, i.e. the AWGN channel, which serves as the lower bound of the Turbo equalization. For $\tau=0.5$, it can be seen that the DL-SPA(15, 2) outperforms the SPDA(15, 2) 1.8 dB at a BER=$10^{-3}$ and the DL-SPA(15, 2) has shown a 1.5 dB performance gain over the SPDA(15, 6) at a BER=$2\times 10^{-3}$. The proposed DL-SPA(15,2) has shown a 0.4 dB gain at a BER=$3\times 10^{-5}$ over the truncated BCJR(15,2). These results imply that the proposed algorithm outperforms the original SPDA and the BCJR algorithm with the same number of considered ISI responses, and reaches the performance of a more complex detection algorithm.

Fig. 7 illustrates the normalized average training loss for every $10^{3}$ training batches for FTN signaling with $\tau=0.6$. Define $\xi_{avg}^{a}$ as the average loss from $(a-1)\times 5\times 10^{3}$ to $a\times 5\times 10^{3}$ batches ($\xi_{avg}^{0}=1$) and $\xi_{cg}^{a}=|(\xi_{avg}^{a}-\xi_{avg}^{a-1})/\xi_{avg}^{a-1}|$ as the percentage of the absolute change on the average loss ($\xi_{cg}^{0}=0$), where $a\in\mathbb{Z}$. Define that a stable performance of the training is reached after $a\times 5\times 10^{3}$ batches, if $\xi_{cg}^{a^{\prime}}<0.1$ for any integer $a^{\prime}>a$. From Fig. 7, the training phase of the proposed algorithm takes roughly $360\times 50000=1.8\times 10^{7}$ samples to converge to a relative stable performance. Benefit from the derived FG and simplified neural network model, this number of training samples is much smaller than the conventional neural network decoders which generally need more than $2^{K}$ ($2^{62}\approx 4.6\times 10^{18}$) training samples to converge to a good performance.

V Conclusion

In this paper, we proposed a deep learning assisted sum-product detection algorithm for FTN signaling. By concatenating a NN to the FG of conventional FTN systems, the proposed detection algorithm computes the a posterior probability with the help of the neural network. A new message updating rule is proposed so that the proposed detection algorithm does not need to be optimized with respect to any particular channel decoder. Furthermore, a simplified CNN architecture for the additional neural network FN is introduced to reduce the training complexity. Simulation results show that the proposed DL-SPA provides a performance gain compared to the SPDA under the same number of ISI responses. Meanwhile, benefiting from the well-developed model of the original factor graph and the simplified structure of the CNN, the DL-SPA needs a much smaller number of batches to train the neural network compared to the conventional neural network decoder which needs at least $2^{K}$ training samples.