Deep Learning for Communication over Dispersive Nonlinear Channels: Performance and Comparison with Classical Digital Signal Processing

TL;DR

This paper demonstrates that deep learning autoencoders, specifically SBRNN, can effectively communicate over dispersive nonlinear optical channels, achieving BER performance comparable to classical methods like MLSD with lower computational complexity.

Contribution

The study introduces a deep learning autoencoder with SBRNN architecture for optical IM/DD communication, showing improved performance and reduced complexity compared to traditional schemes.

Findings

SBRNN autoencoder achieves BER similar to MLSD at 42 Gb/s.

Performance is enhanced by weighted sequence estimation and bit-to-symbol optimization.

Deep learning methods offer comparable results with lower computational complexity.

Abstract

In this paper, we apply deep learning for communication over dispersive channels with power detection, as encountered in low-cost optical intensity modulation/direct detection (IM/DD) links. We consider an autoencoder based on the recently proposed sliding window bidirectional recurrent neural network (SBRNN) design to realize the transceiver for optical IM/DD communication. We show that its performance can be improved by introducing a weighted sequence estimation scheme at the receiver. Moreover, we perform bit-to-symbol mapping optimization to reduce the bit-error rate (BER) of the system. Furthermore, we carry out a detailed comparison with classical schemes based on pulse-amplitude modulation and maximum likelihood sequence detection (MLSD). Our investigation shows that for a reference 42\,Gb/s transmission, the SBRNN autoencoder achieves a BER performance comparable to MLSD, when both systems account for the same amount of memory. In contrast to MLSD, the SBRNN performance is achieved without incurring a computational complexity exponentially growing with the processed memory.