Cross-Attention Message-Passing Transformers for Code-Agnostic Decoding in 6G Networks

TL;DR

This paper introduces a novel transformer-based, code-agnostic decoder for 6G networks that is flexible, scalable, and achieves state-of-the-art performance across various code types without retraining.

Contribution

The paper proposes a new cross-attention message-passing transformer architecture that is invariant to code parameters, enabling unified decoding for multiple code types in 6G networks.

Findings

Achieved state-of-the-art decoding performance among single neural decoders.

Developed a universal model that generalizes across code length, rate, and class.

Enhanced decoding accuracy for short blocklength codes with ensemble methods.

Abstract

Channel coding for 6G networks is expected to support a wide range of requirements arising from heterogeneous communication scenarios. These demands challenge traditional code-specific decoders, which lack the flexibility and scalability required for next-generation systems. To tackle this problem, we propose an AI-native foundation model for unified and code-agnostic decoding based on the transformer architecture. We first introduce a cross-attention message-passing transformer (CrossMPT). CrossMPT employs two masked cross-attention blocks that iteratively update two distinct input representations-magnitude and syndrome vectors-allowing the model to effectively learn the decoding problem. Notably, our CrossMPT has achieved state-of-the-art decoding performance among single neural decoders. Building on this, we develop foundation CrossMPT (FCrossMPT) by making the architecture invariant to code length, rate, and class, allowing a single trained model to decode a broad range of codes without retraining. To further enhance decoding performance, particularly for short blocklength codes, we propose CrossMPT ensemble decoder (CrossED), an ensemble decoder composed of multiple parallel CrossMPT blocks employing different parity-check matrices. This architecture can also serve as a foundation model, showing strong generalization across diverse code types. Overall, the proposed AI-native code-agnostic decoder offers flexibility, scalability, and high performance, presenting a promising direction to channel coding for 6G networks.