Some Extensions of Gallager's Method to General Sources and Channels

TL;DR

This paper extends Gallager's method to general sources and channels within information-spectrum theory, demonstrating its tight bounds and applying it to prove key theorems like Slepian-Wolf and channel coding for broad cases.

Contribution

It generalizes Gallager's bounds to non-memoryless sources and channels, providing a new approach for error exponent estimation in information-spectrum settings.

Findings

Gallager bounds are tight for general sources and channels.

The method proves the direct parts of Slepian-Wolf and channel coding theorems.

Extensions apply to broad classes beyond traditional memoryless models.

Abstract

The Gallager bound is well known in the area of channel coding. However, most discussions about it mainly focus on its applications to memoryless channels. We show in this paper that the bounds obtained by Gallager's method are very tight even for general sources and channels that are defined in the information-spectrum theory. Our method is mainly based on the estimations of error exponents in those bounds, and by these estimations we proved the direct part of the Slepian-Wolf theorem and channel coding theorem for general sources and channels.