An Extended Cencov-Campbell Characterization of Conditional Information Geometry

TL;DR

This paper extends the axiomatic foundation of conditional information geometry, linking it to Fisher geometry and providing new insights into the mathematical underpinnings of logistic regression and AdaBoost.

Contribution

It offers a novel axiomatic characterization of conditional information geometry that generalizes previous work and connects to important machine learning models.

Findings

Provides a new axiomatic interpretation of the primal problems in logistic regression and AdaBoost.

Extends Cencov and Campbell's derivation to the cone of positive conditional models.

Links the conditional I-divergence with the product Fisher information metric.

Abstract

We formulate and prove an axiomatic characterization of conditional information geometry, for both the normalized and the nonnormalized cases. This characterization extends the axiomatic derivation of the Fisher geometry by Cencov and Campbell to the cone of positive conditional models, and as a special case to the manifold of conditional distributions. Due to the close connection between the conditional I-divergence and the product Fisher information metric the characterization provides a new axiomatic interpretation of the primal problems underlying logistic regression and AdaBoost.