Experimental Investigation of Deep Learning for Digital Signal Processing in Short Reach Optical Fiber Communications

TL;DR

This paper experimentally evaluates deep learning methods, specifically RNN-based autoencoders, to enhance performance in short reach optical fiber communications, achieving higher data rates and longer transmission distances.

Contribution

It demonstrates the effectiveness of optimizing RNN-based DSP for optical communications, improving reach and data rates through experimental data-driven neural network parameter tuning.

Findings

Achieved 42 Gb/s transmission with BER below 6.7% at 70 km

Transmitted 84 Gb/s at 20 km

Deep learning DSP improves BER and system reach

Abstract

We investigate methods for experimental performance enhancement of auto-encoders based on a recurrent neural network (RNN) for communication over dispersive nonlinear channels. In particular, our focus is on the recently proposed sliding window bidirectional RNN (SBRNN) optical fiber autoencoder. We show that adjusting the processing window in the sequence estimation algorithm at the receiver improves the reach of simple systems trained on a channel model and applied "as is" to the transmission link. Moreover, the collected experimental data was used to optimize the receiver neural network parameters, allowing to transmit 42 Gb/s with bit-error rate (BER) below the 6.7% hard-decision forward error correction threshold at distances up to 70km as well as 84 Gb/s at 20 km. The investigation of digital signal processing (DSP) optimized on experimental data is extended to pulse amplitude modulation with receivers performing sliding window sequence estimation using a feed-forward or a recurrent neural network as well as classical nonlinear Volterra equalization. Our results show that, for fixed algorithm memory, the DSP based on deep learning achieves an improved BER performance, allowing to increase the reach of the system.