Robust Model Selection for Discovery of Latent Mechanistic Processes

TL;DR

This paper introduces ACDC, a robust model selection method that accurately identifies the number of latent mechanistic processes in models, balancing sensitivity and robustness, and outperforming existing techniques.

Contribution

The paper proposes the ACDC criterion, a novel approach for model selection that combines likelihood sensitivity with nonparametric robustness for latent structure discovery.

Findings

ACDC reliably identifies the true number of latent processes.

It outperforms existing methods in various applications.

ACDC is robustly consistent for matrix factorization and mixture models.

Abstract

When learning interpretable latent structures using model-based approaches, even small deviations from modeling assumptions can lead to inferential results that are not mechanistically meaningful. In this work, we consider latent structures that consist of $K_o$ mechanistic processes, where $K_o$ is unknown. When the model is misspecified, likelihood-based model selection methods can substantially overestimate $K_o$ while more robust nonparametric methods can be overly conservative. Hence, there is a need for approaches that combine the sensitivity of likelihood-based methods with the robustness of nonparametric ones. We formalize this objective in terms of a robust model selection consistency property, which is based on a component-level discrepancy measure that captures the mechanistic structure of the model. We then propose the accumulated cutoff discrepancy criterion (ACDC), which leverages plug-in estimates of component-level discrepancies. To apply ACDC, we develop mechanistically meaningful component-level discrepancies for a general class of latent variable models that includes unsupervised and supervised variants of probabilistic matrix factorization and mixture modeling. We show that ACDC is robustly consistent when applied to unsupervised matrix factorization and mixture models. Numerical results demonstrate that in practice our approach reliably identifies a mechanistically meaningful number of latent processes in numerous illustrative applications, outperforming existing methods.