A Limit Theorem in Singular Regression Problem

TL;DR

This paper establishes a limit theorem for singular regression problems, linking the estimation of true distributions to algebraic invariants, and enabling generalization error estimation without knowing the true distribution.

Contribution

It introduces a limit theorem connecting singular regression to algebraic invariants, aiding in estimating generalization error without true distribution knowledge.

Findings

Proves a limit theorem relating singular regression to algebraic invariants.

Enables estimation of generalization error from training error without true distribution.

Provides a theoretical foundation for statistical analysis in singular models.

Abstract

In statistical problems, a set of parameterized probability distributions is used to estimate the true probability distribution. If Fisher information matrix at the true distribution is singular, then it has been left unknown what we can estimate about the true distribution from random samples. In this paper, we study a singular regression problem and prove a limit theorem which shows the relation between the singular regression problem and two birational invariants, a real log canonical threshold and a singular fluctuation. The obtained theorem has an important application to statistics, because it enables us to estimate the generalization error from the training error without any knowledge of the true probability distribution.