Contextual Memory-Enhanced Source Coding for Low-SNR Communications

TL;DR

This paper introduces a Memory-Augmented Source Coding scheme that enhances the robustness of low-SNR communication systems by internalizing contextual patterns into source models, improving error resilience and coding efficiency.

Contribution

It proposes a novel shared memory-based source model with a routing mechanism to better estimate source probabilities under noisy conditions, advancing modular source-channel coding.

Findings

MASC reduces average codelength in low-SNR channels.

MASC improves robustness against residual channel errors.

Experiments show MASC outperforms existing methods in Rayleigh fading and AWGN channels.

Abstract

While Separate Source-Channel Coding (SSCC) retains the practical benefits of modular system design, its effectiveness in noisy text transmission is fundamentally constrained by the fragility of autoregressive source decoding. In low-SNR regimes, even a small number of residual bit errors after channel decoding may derail the subsequent lossless reconstruction process, especially when Arithmetic Coding (AC) relies on Large Language Model (LLM)-based probability estimation. Existing remedies either strengthen channel decoding based solely on channel observations or introduce contextual information only at the receiver for post-hoc correction, yet neither fully addresses the fragility of source probability modeling under residual channel errors. To this end, this paper proposes a Memory-Augmented Source Coding (MASC) scheme for robust SSCC-based transmission. Rather than treating context as external side information, MASC internalizes contextual patterns into a source model shared by both the transmitter-side source encoder and the receiver-side source decoder. Specifically, MASC employs a shared Parameterized Contextual Memory (PCM) to encode multi-order $n$-gram patterns, and further introduces a Mixture-of-Memory-Experts Router (MMER) to perform sparse, hidden-state-dependent routing over memory experts during autoregressive source modeling. By adaptively activating only the most relevant memories at each coding step, MASC refines source probability estimation, shortens average codelength, and mitigates the sensitivity of source decoding to residual channel errors. Extensive experiments over Rayleigh fading and AWGN channels demonstrate the effectiveness of the proposed scheme compared with state-of-the-art methods.