How to Mask in Error Correction Code Transformer: Systematic and Double Masking

TL;DR

This paper introduces systematic and double masking techniques for Error Correction Code Transformers (ECCT) to improve decoding performance and reduce complexity, achieving state-of-the-art results in neural ECC decoding.

Contribution

It proposes a novel systematic masking matrix and a double-masked ECCT architecture, enhancing neural decoder performance over traditional ECCTs.

Findings

Double-masked ECCT outperforms conventional ECCT.

Proposed methods achieve state-of-the-art decoding performance.

Significant performance improvements demonstrated through extensive simulations.

Abstract

In communication and storage systems, error correction codes (ECCs) are pivotal in ensuring data reliability. As deep learning's applicability has broadened across diverse domains, there is a growing research focus on neural network-based decoders that outperform traditional decoding algorithms. Among these neural decoders, Error Correction Code Transformer (ECCT) has achieved the state-of-the-art performance, outperforming other methods by large margins. To further enhance the performance of ECCT, we propose two novel methods. First, leveraging the systematic encoding technique of ECCs, we introduce a new masking matrix for ECCT, aiming to improve the performance and reduce the computational complexity. Second, we propose a novel transformer architecture of ECCT called a double-masked ECCT. This architecture employs two different mask matrices in a parallel manner to learn more diverse features of the relationship between codeword bits in the masked self-attention blocks. Extensive simulation results show that the proposed double-masked ECCT outperforms the conventional ECCT, achieving the state-of-the-art decoding performance with significant margins.