Asymptotic Behavior of Bayesian Generalization Error in Multinomial Mixtures

TL;DR

This paper investigates the asymptotic behavior of Bayesian generalization error in multinomial mixture models, which are singular and non-identifiable, by applying algebraic geometric methods to clarify their mathematical properties.

Contribution

It provides the first analysis of the real log canonical thresholds and multiplicities of multinomial mixtures, elucidating their asymptotic behaviors using algebraic geometry.

Findings

Determined the real log canonical thresholds of multinomial mixtures.

Analyzed the asymptotic behavior of generalization error and free energy.

Clarified the mathematical properties of singular models using algebraic geometry.

Abstract

Multinomial mixtures are widely used in the information engineering field, however, their mathematical properties are not yet clarified because they are singular learning models. In fact, the models are non-identifiable and their Fisher information matrices are not positive definite. In recent years, the mathematical foundation of singular statistical models are clarified by using algebraic geometric methods. In this paper, we clarify the real log canonical thresholds and multiplicities of the multinomial mixtures and elucidate their asymptotic behaviors of generalization error and free energy.