A Bit of Information Theory, and the Data Augmentation Algorithm Converges

TL;DR

This paper uses information theory to prove a convergence theorem for the data augmentation algorithm, enhancing understanding of its theoretical foundations.

Contribution

It introduces a novel information-theoretic approach to establish convergence of the data augmentation algorithm.

Findings

Proves convergence of the DA algorithm using information theory

Provides a new theoretical framework for analyzing DA algorithms

Enhances understanding of DA algorithm stability

Abstract

The data augmentation (DA) algorithm is a simple and powerful tool in statistical computing. In this note basic information theory is used to prove a nontrivial convergence theorem for the DA algorithm.