LLM-Viterbi: Semantic-Aware Decoding for Convolutional Codes

TL;DR

This paper introduces an LLM-augmented Viterbi decoder that leverages linguistic structure to improve error correction in text transmission over noisy channels, outperforming traditional methods.

Contribution

It presents a novel semantic-aware decoding framework integrating large language models into Viterbi decoding for the first time.

Findings

Achieves 1.5 dB more coding gain in BLER for convolutional codes.

Over 50% improvement in semantic similarity.

Significant performance gains over conventional Viterbi decoding.

Abstract

Traditional wireless communications rely solely on bit-level channel coding for error correction, without exploiting the inherent linguistic structure of the data source. This paper proposes a large language model (LLM) Viterbi decoder that integrates LLM priors into the Viterbi decoding for text transmission over AWGN channels. The proposed decoder maintains multiple candidate paths during the Viterbi decoding and periodically evaluates path reliabilities using a fine-tuned Byte-level T5 (ByT5) language model. By combining channel reliability metrics with semantic probability from the LLM, it outputs the path that maximizes the joint likelihood of channel observations and linguistic coherence. Simulations show that our decoder achieves significant performance gains over conventional Viterbi decoding in terms of both block error rate (BLER) and semantic similarity. For convolutional codes with constraint length 3, it achieves approximately 1.5 dB more coding gain in BLER, with over 50% improvements in semantic similarity. The framework can extend to other structured data sources beyond text.