Notion of information and independent component analysis

TL;DR

This paper explores the theoretical foundations of information measures, non-Gaussianity, and their roles in independent component analysis, emphasizing the significance of Gaussian and uniform distributions in continuous variables.

Contribution

It provides a detailed discussion of the connections between information, non-Gaussianity, and independence, and examines their application as projection indices in ICA.

Findings

Information measures relate to non-Gaussianity in ICA.

Gaussian and uniform distributions have special roles in information measures.

Third and fourth cumulants serve as non-Gaussianity indices.

Abstract

Partial orderings and measures of information for continuous univariate random variables with special roles of Gaussian and uniform distributions are discussed. The information measures and measures of non-Gaussianity including third and fourth cumulants are generally used as projection indices in the projection pursuit approach for the independent component analysis. The connections between information, non-Gaussianity and statistical independence in the context of independent component analysis is discussed in detail.