Maximum Ideal Likelihood Estimation: A Unified Inference Framework for Latent Variable Models

TL;DR

This paper introduces Maximum Ideal Likelihood Estimation (MILE), a unified framework for latent variable models that directly uses the complete data distribution, offering robustness and efficiency where traditional methods struggle.

Contribution

The paper proposes MILE, a novel estimation approach that directly exploits the joint distribution of complete data, providing a flexible alternative to traditional latent variable inference methods.

Findings

MILE is consistent and asymptotically normal under mild conditions.

MILE outperforms existing methods in computational efficiency and scalability.

Simulations and real-data applications demonstrate MILE's empirical advantages.

Abstract

This paper develops a unified estimation framework, the Maximum Ideal Likelihood Estimation (MILE), for general parametric models with latent variables. Unlike traditional approaches relying on the marginal likelihood of the observed data, MILE directly exploits the joint distribution of the complete data by treating the latent variables as parameters (the ideal likelihood). Borrowing strength from optimisation techniques and algorithms, MILE is a broadly applicable framework in case that traditional methods fail, such as when the marginal likelihood has non-finite expectations. MILE offers a flexible and robust alternative to established techniques, including the Expectation-Maximisation algorithm and Markov chain Monte Carlo. We facilitate statistical inference of MILE on consistency, asymptotic distribution, and equivalence to the Maximum Likelihood Estimation, under some mild conditions. Extensive simulations illustrative real-data applications illustrate the empirical advantages of MILE, outperforming existing methods on computational feasibility and scalability.