Replacing the Soft FEC Limit Paradigm in the Design of Optical Communication Systems

TL;DR

This paper challenges the traditional FEC limit paradigm in optical communication system design, demonstrating that it can underestimate spectral efficiencies and proposing generalized mutual information as a better predictor for post-FEC BER.

Contribution

The paper introduces the invalidity of the soft FEC limit paradigm for soft-decision decoding and advocates for using generalized mutual information for more accurate BER prediction.

Findings

Soft FEC limit underestimates spectral efficiencies by up to 20%.

Generalized mutual information provides consistent BER predictions.

Experimental results confirm theoretical analysis across transmission regimes.

Abstract

The FEC limit paradigm is the prevalent practice for designing optical communication systems to attain a certain bit-error rate (BER) without forward error correction (FEC). This practice assumes that there is an FEC code that will reduce the BER after decoding to the desired level. In this paper, we challenge this practice and show that the concept of a channel-independent FEC limit is invalid for soft-decision bit-wise decoding. It is shown that for low code rates and high order modulation formats, the use of the soft FEC limit paradigm can underestimate the spectral efficiencies by up to 20%. A better predictor for the BER after decoding is the generalized mutual information, which is shown to give consistent post-FEC BER predictions across different channel conditions and modulation formats. Extensive optical full-field simulations and experiments are carried out in both the linear and nonlinear transmission regimes to confirm the theoretical analysis.