Theoretical Grounding for Estimation in Conditional Independence Multivariate Finite Mixture Models

TL;DR

This paper introduces a new theoretical framework and algorithm for nonparametric estimation of multivariate finite mixture models under conditional independence, providing proofs of existence and novel properties.

Contribution

It presents a new formulation of the estimation problem, a novel projection-multiplication operator, and proves the existence of solutions, advancing the theoretical understanding of mixture model estimation.

Findings

Proposed a penalized smoothed Kullback-Leibler objective function.

Derived the NSMM algorithm with a new projection-multiplication operator.

Proved the existence of solutions to the optimization problem.

Abstract

For the nonparametric estimation of multivariate finite mixture models with the conditional independence assumption, we propose a new formulation of the objective function in terms of penalized smoothed Kullback-Leibler distance. The nonlinearly smoothed majorization-minimization (NSMM) algorithm is derived from this perspective. An elegant representation of the NSMM algorithm is obtained using a novel projection-multiplication operator, a more precise monotonicity property of the algorithm is discovered, and the existence of a solution to the main optimization problem is proved for the first time.