The dually flat structure for singular models

TL;DR

This paper extends the dually flat structure in information geometry to singular models with degenerate metrics, enabling broader applications in physics and deep learning by generalizing key theorems.

Contribution

It introduces a quasi-Hessian manifold framework that generalizes statistical manifolds to handle singular models with degenerate metrics and symmetric cubic tensors.

Findings

Extended Pythagorean theorem for singular models

Projection theorem validity in generalized setup

Applicability to deep learning and mathematical physics

Abstract

The dually flat structure introduced by Amari-Nagaoka is highlighted in information geometry and related fields. In practical applications, however, the underlying pseudo-Riemannian metric may often be degenerate, and such an excellent geometric structure is rarely defined on the entire space. To fix this trouble, in the present paper, we propose a novel generalization of the dually flat structure for a certain class of singular models from the viewpoint of Lagrange and Legendre singularity theory - we introduce a quasi-Hessian manifold endowed with a possibly degenerate metric and a particular symmetric cubic tensor, which exceeds the concept of statistical manifolds and is adapted to the theory of (weak) contrast functions. In particular, we establish Amari-Nagaoka's extended Pythagorean theorem and projection theorem in this general setup, and consequently, most of applications of these theorems are suitably justified even for such singular cases. This work is motivated by various interests with different backgrounds from Frobenius structure in mathematical physics to Deep Learning in data science.