Independent Component Analysis Over Galois Fields

TL;DR

This paper extends Independent Component Analysis to Galois fields, establishing identifiability conditions, proposing two algorithms, and analyzing their performance, with simulations demonstrating their effectiveness in finite fields.

Contribution

It introduces ICA over Galois fields, derives identifiability conditions, and proposes two novel algorithms for source separation in this algebraic setting.

Findings

Identifiability requires sources not to be uniformly distributed.

Pairwise independence implies full independence for P=2,3 but not for P>3.

Algorithms show promising performance in simulations for binary and higher-order fields.

Abstract

We consider the framework of Independent Component Analysis (ICA) for the case where the independent sources and their linear mixtures all reside in a Galois field of prime order P. Similarities and differences from the classical ICA framework (over the Real field) are explored. We show that a necessary and sufficient identifiability condition is that none of the sources should have a Uniform distribution. We also show that pairwise independence of the mixtures implies their full mutual independence (namely a non-mixing condition) in the binary (P=2) and ternary (P=3) cases, but not necessarily in higher order (P>3) cases. We propose two different iterative separation (or identification) algorithms: One is based on sequential identification of the smallest-entropy linear combinations of the mixtures, and is shown to be equivariant with respect to the mixing matrix; The other is based on sequential minimization of the pairwise mutual information measures. We provide some basic performance analysis for the binary (P=2) case, supplemented by simulation results for higher orders, demonstrating advantages and disadvantages of the proposed separation approaches.