An Asymptotic Equation Linking WAIC and WBIC in Singular Models

TL;DR

This paper derives an asymptotic equation linking WAIC and WBIC in singular models, clarifying their relationship and asymptotic behavior, which enhances understanding and computational efficiency in model selection.

Contribution

It provides the first theoretical derivation connecting WAIC and WBIC in singular models, deepening the understanding of their asymptotic relationship.

Findings

An asymptotic equation linking WAIC and WBIC was derived.

The equation provides an unbiased approximation of WAIC using WBIC's posterior.

The results clarify the structural relationship between WAIC and WBIC in singular learning theory.

Abstract

In statistical learning, models are classified as regular or singular depending on whether the mapping from parameters to probability distributions is injective. Most models with hierarchical structures or latent variables are singular, for which conventional criteria such as the Akaike Information Criterion and the Bayesian Information Criterion are inapplicable due to the breakdown of normal approximations for the likelihood and posterior. To address this, the Widely Applicable Information Criterion (WAIC) and the Widely Applicable Bayesian Information Criterion (WBIC) have been proposed. Since WAIC and WBIC are computed using posterior distributions at different temperature settings, separate posterior sampling is generally required. In this paper, we theoretically derive an asymptotic equation that links WAIC and WBIC, despite their dependence on different posteriors. This equation yields an asymptotically unbiased expression of WAIC in terms of the posterior distribution used for WBIC. The result clarifies the structural relationship between these criteria within the framework of singular learning theory, and deepens understanding of their asymptotic behavior. This theoretical contribution provides a foundation for future developments in the computational efficiency of model selection in singular models.