# Deep Learning Assisted Sum-Product Detection Algorithm for Faster-than-Nyquist Signaling

## TL;DR

This paper introduces a deep learning aided sum-product detection algorithm for faster-than-Nyquist signaling, significantly improving detection performance by mitigating residual intersymbol interference with a neural network-enhanced factor graph approach.

## Contribution

It proposes a novel neural network integrated sum-product detection algorithm that enhances FTN signaling detection by effectively handling residual ISI and enabling turbo equalization.

## Key findings

- Achieves up to 2.5 dB performance gain over conventional methods.
- Uses a simplified convolutional neural network requiring minimal training batches.
- Demonstrates improved bit error rate performance in simulations.

## Abstract

A deep learning assisted sum-product detection algorithm (DL-SPA) for faster-than-Nyquist (FTN) signaling is proposed in this paper. The proposed detection algorithm concatenates a neural network to the variable nodes of the conventional factor graph of the FTN system to help the detector converge to the a posterior probabilities based on the received sequence. More specifically, the neural network performs as a function node in the modified factor graph to deal with the residual intersymbol interference (ISI) that is not modeled by the conventional detector with a limited number of ISI taps. We modify the updating rule in the conventional sum-product algorithm so that the neural network assisted detector can be complemented to a Turbo equalization. Furthermore, a simplified convolutional neural network is employed as the neural network function node to enhance the detector's performance and the neural network needs a small number of batches to be trained. Simulation results have shown that the proposed DL-SPA achieves a performance gain up to 2.5 dB with the same bit error rate compared to the conventional sum-product detection algorithm under the same ISI responses.