An Integral Representation of the Logarithmic Function with Applications in Information Theory

TL;DR

This paper investigates an integral representation of the logarithmic function and demonstrates its utility in deriving exact formulas for expectations involving logarithms, with applications in information theory such as data compression and channel capacity.

Contribution

It introduces a rigorous integral representation of the logarithm and applies it to various information-theoretic problems, offering an alternative to the replica method.

Findings

Derived compact formulas for expectations of logarithms of random variables

Applied the integral representation to entropy and capacity calculations

Provided a rigorous mathematical tool for information theory applications

Abstract

We explore a well-known integral representation of the logarithmic function, and demonstrate its usefulness in obtaining compact, easily-computable exact formulas for quantities that involve expectations and higher moments of the logarithm of a positive random variable (or the logarithm of a sum of positive random variables). The integral representation of the logarithm is proved useful in a variety of information-theoretic applications, including universal lossless data compression, entropy and differential entropy evaluations, and the calculation of the ergodic capacity of the single-input, multiple-output (SIMO) Gaussian channel with random parameters (known to both transmitter and receiver). This integral representation and its variants are anticipated to serve as a useful tool in additional applications, as a rigorous alternative to the popular (but non-rigorous) replica method (at least in some situations).