Complexity Reduction over Bi-RNN-Based Nonlinearity Mitigation in Dual-Pol Fiber-Optic Communications via a CRNN-Based Approach

TL;DR

This paper introduces a more efficient neural network architecture combining CNN encoders and unidirectional RNNs for nonlinearity mitigation in dual-polarization fiber-optic communications, achieving similar performance to bi-RNNs with significantly reduced complexity.

Contribution

The paper proposes a novel hybrid CNN-RNN architecture that reduces computational complexity while maintaining state-of-the-art nonlinearity mitigation performance in fiber-optic systems.

Findings

Achieves >50% lower complexity than bi-RNN methods.

Maintains comparable bit error rates to existing bi-RNN approaches.

Effective in dual-polarization 16-QAM 64 GBd fiber-optic transmission.

Abstract

Bidirectional recurrent neural networks (bi-RNNs), in particular, bidirectional long short term memory (bi-LSTM), bidirectional gated recurrent unit, and convolutional bi-LSTM models have recently attracted attention for nonlinearity mitigation in fiber-optic communication. The recently adopted approaches based on these models, however, incur a high computational complexity which may impede their real-time functioning. In this paper, by addressing the sources of complexity in these methods, we propose a more efficient network architecture, where a convolutional neural network encoder and a unidirectional many-to-one vanilla RNN operate in tandem, each best capturing one set of channel impairments while compensating for the shortcomings of the other. We deploy this model in two different receiver configurations. In one, the neural network is placed after a linear equalization chain and is merely responsible for nonlinearity mitigation; in the other, the neural network is directly placed after the chromatic dispersion compensation and is responsible for joint nonlinearity and polarization mode dispersion compensation. For a 16-QAM 64 GBd dual-polarization optical transmission over 14x80 km standard single-mode fiber, we demonstrate that the proposed hybrid model achieves the bit error probability of the state-of-the-art bi-RNN-based methods with greater than 50% lower complexity, in both receiver configurations.