Ultralow complexity long short-term memory network for fiber nonlinearity mitigation in coherent optical communication systems

TL;DR

This paper introduces a low-complexity neural network-based equalizer, Co-LSTM, for fiber nonlinearity mitigation in optical systems, achieving comparable performance to traditional methods but with significantly reduced computational complexity.

Contribution

The paper proposes a novel Co-LSTM neural network with a recycling mechanism that drastically reduces complexity for fiber nonlinearity mitigation in optical communications.

Findings

Co-LSTM achieves similar nonlinear mitigation performance as DBP.

Complexity of Co-LSTM is nearly independent of transmission distance.

Co-LSTM offers an attractive low-complexity alternative for optical fiber nonlinearity mitigation.

Abstract

Fiber Kerr nonlinearity is a fundamental limitation to the achievable capacity of long-distance optical fiber communication. Digital back-propagation (DBP) is a primary methodology to mitigate both linear and nonlinear impairments by solving the inverse-propagating nonlinear Schr\"odinger equation (NLSE), which requires detailed link information. Recently, the paradigms based on neural network (NN) were proposed to mitigate nonlinear transmission impairments in optical communication systems. However, almost all neural network-based equalization schemes yield high computation complexity, which prevents the practical implementation in commercial transmission systems. In this paper, we propose a center-oriented long short-term memory network (Co-LSTM) incorporating a simplified mode with a recycling mechanism in the equalization operation, which can mitigate fiber nonlinearity in coherent optical communication systems with ultralow complexity. To validate the proposed methodology, we carry out an experiment of ten-channel wavelength division multiplexing (WDM) transmission with 64 Gbaud polarization-division-multiplexed 16-ary quadrature amplitude modulation (16-QAM) signals. Co-LSTM and DBP achieve a comparable performance of nonlinear mitigation. However, the complexity of Co-LSTM with a simplified mode is almost independent of the transmission distance, which is much lower than that of the DBP. The proposed Co-LSTM methodology presents an attractive approach for low complexity nonlinearity mitigation with neural networks.