In-Context Source and Channel Coding

TL;DR

This paper introduces an In-Context Decoding framework that improves the robustness of separate source-channel coding for text transmission, especially under low SNR conditions, by leveraging an error correction transformer and LLM-based decoding.

Contribution

It proposes a novel receiver-side decoding method that enhances SSCC robustness without changing the transmitter, using an error correction transformer and confidence-based candidate selection.

Findings

Significant performance gains over traditional SSCC in AWGN and Rayleigh channels.

Theoretical guarantees on stability and convergence of the decoding process.

Improved resilience against residual bit errors in low SNR regimes.

Abstract

Separate Source-Channel Coding (SSCC) remains attractive for text transmission due to its modularity and compatibility with mature entropy coders and powerful channel codes. However, SSCC often suffers from a pronounced cliff effect in low Signal-to-Noise Ratio (SNR) regimes, where residual bit errors after channel decoding can catastrophically break lossless source decoding, especially for Arithmetic Coding (AC) driven by Large Language Models (LLMs). This paper proposes a receiver-side In-Context Decoding (ICD) framework that enhances SSCC robustness without modifying the transmitter. ICD leverages an Error Correction Code Transformer (ECCT) to obtain bit-wise reliability for the decoded information bits. Based on the context-consistent bitstream, ICD constructs a confidence-ranked candidate pool via reliability-guided bit flipping, samples a compact yet diverse subset of candidates, and applies an LLM-based arithmetic decoder to obtain both reconstructions and sequence-level log-likelihoods. A reliability-likelihood fusion rule then selects the final output. We further provide theoretical guarantees on the stability and convergence of the proposed sampling procedure. Extensive experiments over Additive White Gaussian Noise (AWGN) and Rayleigh fading channels demonstrate consistent gains compared with conventional SSCC baselines and representative Joint Source-Channel Coding (JSCC) schemes.