Orthogonal Decomposition of Discretization-Induced Transport-Information Cost under Rank-Deficient Parametrizations

TL;DR

This paper develops a geometric framework to analyze discretization-induced transport-information costs, especially under rank-deficient parametrizations, by orthogonally decomposing covariance matrices to distinguish observable and unobservable components.

Contribution

It introduces an orthogonal decomposition method for covariance matrices to separate discretization costs into observable and unobservable parts under rank-deficient parametrizations.

Findings

Decomposition separates observable and unobservable discretization costs.

Framework clarifies the role of parametrization-dependent information loss.

Provides insight into partial observability of transport-information costs.

Abstract

When we consider discretization of continuous probability distributions, it inevitably induces irreversible geometric distortion of local measure on the discretized support. While such discretziation-induced distortion is extrinsic to information geometry (IG) alone, we recently demonstrate that the discretization cost can be naturally characterized by the standard Kullback-Leibler (KL) divergence between continuous distributions as expectation of their infinitesimal parameter variations. The framework is based on the correspondence between optimal transport (OT) and IG, primarily requring the selected parameters directly identifiable with support coordinates. The present work extends the framework to more generalized parametrization theta, particularly the Jacobian between theta and support coordinates is rank-deficient, which generally results in breaking down the interpretation of the discretization-induced costs as information-geometric quantities. To address the problem, we here introduce an orthogonal decomposition of the second-moment tensor onto linear subspace for the covariance matrices generated by parameter fluctuations, based on Frobenius projection. The decomposition naturally separates the discretization cost into observable and unobservable components relative to the chosen parametrization. The present formulation provides a geometric framework for analyzing partial observability of discretization-induced transport-information costs and clarifies the role of parametrization-dependent information loss.