Neural Turbo Equalization: Deep Learning for Fiber-Optic Nonlinearity Compensation

TL;DR

This paper introduces a deep neural network-based turbo equalizer for fiber-optic communication systems, effectively mitigating nonlinearity and improving throughput and achievable rates through optimized coding and EXIT analysis.

Contribution

It proposes a novel DNN-based turbo equalization framework using ResNet, enhancing fiber nonlinearity mitigation and decoding convergence in optical communications.

Findings

Achieves a throughput gain of 0.61 b/s/Hz with DNN-TEQ.

Improves achievable rates by up to 0.12 b/s/Hz with optimized LDPC codes.

Verifies the effectiveness of the proposed method through EXIT analysis.

Abstract

Recently, data-driven approaches motivated by modern deep learning have been applied to optical communications in place of traditional model-based counterparts. The application of deep neural networks (DNN) allows flexible statistical analysis of complicated fiber-optic systems without relying on any specific physical models. Due to the inherent nonlinearity in DNN, various equalizers based on DNN have shown significant potentials to mitigate fiber nonlinearity. In this paper, we propose a turbo equalization (TEQ) based on DNN as a new alternative framework to deal with nonlinear fiber impairments for future coherent optical communications. The proposed DNN-TEQ is constructed with nested deep residual networks (ResNet) to train extrinsic likelihood given soft-information feedback from channel decoding. Through extrinsic information transfer (EXIT) analysis, we verify that our DNN-TEQ can accelerate decoding convergence to achieve a significant gain in achievable throughput by 0.61b/s/Hz. We also demonstrate that optimizing irregular low-density parity-check (LDPC) codes to match EXIT chart of the DNN-TEQ can improve achievable rates by up to 0.12 b/s/Hz.