On the Identifiability of Finite Mixtures of Finite Product Measures

TL;DR

This paper investigates the conditions under which finite mixture models of finite product measures are identifiable, establishing thresholds based on the number of separable variables and introducing a polynomial-based proof method.

Contribution

It provides necessary and sufficient conditions for identifiability in finite mixture models using separability concepts and polynomial techniques, with tight bounds proven.

Findings

Identifiability if ≥ 2K-1 strongly separable variables, independent of M.

Identifiability if ≥ 2K weakly separable variables.

Counterexamples for fewer than 2K-1 strongly separable variables.

Abstract

The problem of identifiability of finite mixtures of finite product measures is studied. A mixture model with $K$ mixture components and $L$ observed variables is considered, where each variable takes its value in a finite set with cardinality $M$.The variables are independent in each mixture component. The identifiability of a mixture model means the possibility of attaining the mixture components parameters by observing its mixture distribution. In this paper, we investigate fundamental relations between the identifiability of mixture models and the separability of their observed variables by introducing two types of separability: strongly and weakly separable variables. Roughly speaking, a variable is said to be separable, if and only if it has some differences among its probability distributions in different mixture components. We prove that mixture models are identifiable if the number of strongly separable variables is greater than or equal to $2K-1$, independent form $M$. This fundamental threshold is shown to be tight, where a family of non-identifiable mixture models with less than $2K-1$ strongly separable variables is provided. We also show that mixture models are identifiable if they have at least $2K$ weakly separable variables. To prove these theorems, we introduce a particular polynomial, called characteristic polynomial, which translates the identifiability conditions to identity of polynomials and allows us to construct an inductive proof.