New Non-asymptotic Random Channel Coding Theorems

TL;DR

This paper introduces new non-asymptotic coding theorems for discrete channels, providing tight bounds on error probability and rate for finite block lengths, useful for practical communication systems.

Contribution

It develops novel non-asymptotic achievability bounds based on Gallager and Shannon ensembles, with explicit computability and tightness for finite block lengths.

Findings

Bounds are asymptotically tight up to second order

Numerical results show favorable comparison with existing bounds

Bounds are applicable for practical finite block length scenarios

Abstract

New non-asymptotic random coding theorems (with error probability $\epsilon$ and finite block length $n$) based on Gallager parity check ensemble and Shannon random code ensemble with a fixed codeword type are established for discrete input arbitrary output channels. The resulting non-asymptotic achievability bounds, when combined with non-asymptotic equipartition properties developed in the paper, can be easily computed. Analytically, these non-asymptotic achievability bounds are shown to be asymptotically tight up to the second order of the coding rate as $n$ goes to infinity with either constant or sub-exponentially decreasing $\epsilon$. Numerically, they are also compared favourably, for finite $n$ and $\epsilon$ of practical interest, with existing non-asymptotic achievability bounds in the literature in general.