Recent Advances in Algebraic Geometry and Bayesian Statistics

TL;DR

This review discusses recent algebraic geometry methods that have advanced Bayesian statistics, especially for nonidentifiable models with singularities, enabling better analysis and applications in data science and AI.

Contribution

The paper introduces new algebraic geometry-based frameworks and solutions for analyzing nonidentifiable Bayesian models with singularities, enhancing statistical understanding and practical applications.

Findings

Resolution maps find appropriate parameter spaces.

Asymptotic free energy derived for models.

Universal formula linking loss, validation, and information criteria.

Abstract

This article is a review of theoretical advances in the research field of algebraic geometry and Bayesian statistics in the last two decades. Many statistical models and learning machines which contain hierarchical structures or latent variables are called nonidentifiable, because the map from a parameter to a statistical model is not one-to-one. In nonidentifiable models, both the likelihood function and the posterior distribution have singularities in general, hence it was difficult to analyze their statistical properties. However, from the end of the 20th century, new theory and methodology based on algebraic geometry have been established which enables us to investigate such models and machines in the real world. In this article, the following results in recent advances are reported. First, we explain the framework of Bayesian statistics and introduce a new perspective from the birational geometry. Second, two mathematical solutions are derived based on algebraic geometry. An appropriate parameter space can be found by a resolution map, which makes the posterior distribution be normal crossing and the log likelihood ratio function be well-defined. Third, three applications to statistics are introduced. The posterior distribution is represented by the renormalized form, the asymptotic free energy is derived, and the universal formula among the generalization loss, the cross validation, and the information criterion is established. Two mathematical solutions and three applications to statistics based on algebraic geometry reported in this article are now being used in many practical fields in data science and artificial intelligence.