The generalized likelihood decoder: random coding and expurgated bounds

TL;DR

This paper introduces a generalized likelihood decoder framework that encompasses mismatched, universal, and deterministic decoders, providing exact random coding exponents and an improved expurgated bound for stochastic decoding.

Contribution

It develops a direct analysis method for the generalized likelihood decoder, deriving exact exponents and extending results to source-channel coding with side information.

Findings

Exact random coding exponent derived for the generalized likelihood decoder.

Extension of results to source and channel coding with side information.

An expurgated exponent for the stochastic likelihood decoder that can outperform classical bounds.

Abstract

The likelihood decoder is a stochastic decoder that selects the decoded message at random, using the posterior distribution of the true underlying message given the channel output. In this work, we study a generalized version of this decoder where the posterior is proportional to a general function that depends only on the joint empirical distribution of the output vector and the codeword. This framework allows both mismatched versions and universal (MMI) versions of the likelihood decoder, as well as the corresponding ordinary deterministic decoders, among many others. We provide a direct analysis method that yields the exact random coding exponent (as opposed to separate upper bounds and lower bounds that turn out to be compatible, which were derived earlier by Scarlett et al. We also extend the result from pure channel coding to combined source and channel coding (random binning followed by random channel coding) with side information available to the decoder. Finally, returning to pure channel coding, we derive also an expurgated exponent for the stochastic likelihood decoder, which turns out to be at least as tight (and in some cases, strictly so) as the classical expurgated exponent of the maximum likelihood decoder, even though the stochastic likelihood decoder is suboptimal.