Normalized Maximum Likelihood Code-Length on Riemannian Data Spaces

TL;DR

This paper introduces a coordinate-invariant Normalized Maximum Likelihood (NML) tailored for Riemannian manifolds, enabling model selection and regret minimization on non-Euclidean data spaces like hyperbolic spaces.

Contribution

It defines a new Riemannian NML (Rm-NML) that respects geometric structures, extends computational methods, and simplifies calculations on symmetric spaces.

Findings

Derived Rm-NML for hyperbolic spaces

Extended NML computation techniques to Riemannian manifolds

Explicitly computed Rm-NML for normal distributions on hyperbolic spaces

Abstract

In recent years, with the large-scale expansion of graph data, there has been an increased focus on Riemannian manifold data spaces other than Euclidean space. In particular, the development of hyperbolic spaces has been remarkable, and they have high expressive power for graph data with hierarchical structures. Normalized Maximum Likelihood (NML) is employed in regret minimization and model selection. However, existing formulations of NML have been developed primarily in Euclidean spaces and are inherently dependent on the choice of coordinate systems, making it non-trivial to extend NML to Riemannian manifolds. In this study, we define a new NML that reflects the geometric structure of Riemannian manifolds, called the Riemannian manifold NML (Rm-NML). This Rm-NML is invariant under coordinate transformations and coincides with the conventional NML under the natural parameterization in Euclidean space. We extend existing computational techniques for NML to the setting of Riemannian manifolds. Furthermore, we derive a method to simplify the computation of Rm-NML on Riemannian symmetric spaces, which encompass data spaces of growing interest such as hyperbolic spaces. To illustrate the practical application of our proposed method, we explicitly computed the Rm-NML for normal distributions on hyperbolic spaces.

Equations

$$g_p:T_p\mathcal{M}\times T_p\mathcal{M}\to \mathbb{R}.$$

$$g_{ij}(p)=g_p\left({\left(\frac{\partial }{\partial x_i}\right)}_p,{\left(\frac{\partial }{\partial x_j}\right)}_p\right).$$

$$g=\sum _{i,j}{g}_{ij}\mathrm{d}{x}_i\otimes \mathrm{d}{x}_j.$$

$${\mathcal{H}}_p^D\stackrel{\mathrm{def}}{=}\left\{x\mid x\in {\mathbb{R}}^D,x_1^2+\cdots +x_D^2<1\right\}.$$

$$g\stackrel{\mathrm{def}}{=}4\frac{d{x}_1^2+\cdots +d{x}_D^2}{(1-x_1^2-\cdots -x_D^2{)}^2}.$$

$$J_{D}\stackrel{{\scriptstyle\text{ def }}}{{=}}\left[\begin{array}[]{cccc}-1&0&\ldots&0\\ 0&1&&\\ \vdots&&\ddots&\\ 0&&&1\end{array}\right],$$

$$⟨x,y{⟩}_{\mathcal{L}}\stackrel{\text{ def }}{=}{x}^{\top }{J}_{D}y,$$

$${\mathcal{H}}_l^D\stackrel{\text{ def }}{=}\left\{x={\left(x_0,\cdots ,x_D\right)}^T\mid x\in {\mathbb{R}}^{D+1},⟨x,x{⟩}_{\mathcal{L}}=-1,x_0>0\right\}.$$

$$g\stackrel{\mathrm{def}}{=}-(d{x}_0{)}^2+d{x}_1^2+\cdots +d{x}_D^2.$$

$$d\mathrm{vol}(x)\stackrel{\mathrm{def}}{=}\sqrt{\det g_{\mathcal{M}}(x)}dx.$$

$$\sqrt{\det g_{\mathcal{M}}(x^n)}$$

$$\stackrel{\mathrm{def}}{=}$$

$$p(x^n\mid \theta )=p_{\mathrm{vol}}(x^n\mid \theta )\sqrt{\det g_{\mathcal{M}}(x^n)}.$$

$$p_{\mathrm{vol}}(x\mid \mu ,\sigma )\stackrel{\mathrm{def}}{=}\frac{1}{\xi (\sigma )}\exp \left(-\frac{1}{2{\sigma }^2}{d}^2(x,\mu )\right),$$

$$\xi (\sigma )\stackrel{\text{ def }}{=}{\int }_{\mathcal{M}}\exp \left(-\frac{1}{2{\sigma }^2}{d}^2(x,\mu )\right)d\mathrm{vol}(x).$$

$$p_{\mathrm{NML}}(x^n)=\frac{p(x^n\mid \hat{\theta }(x^n))}{{\int }_{y^n\in {\mathcal{X}}^n}p(y^n\mid \hat{\theta }(y^n))d{y}^n}.$$

$$\mathcal{C}(\mathcal{P})\stackrel{\text{ def }}{=}{\int }_{y^n\in {\mathcal{X}}^n}p(y^n\mid \hat{\theta }(y^n))d{y}^n.$$

$${\mathcal{L}}_{\mathrm{NML}}(x^n\mid \mathcal{P})$$

$$=-\log p(x^n\mid \hat{\theta }(x^n))+\log \mathcal{C}(\mathcal{P}).$$

$$p_{\mathrm{NML}}(\cdot )$$

$$=\underset{q}{\mathrm{argmin}}\underset{x^n}{\max }\left\{-\log q(x^n)-\underset{\theta }{\min }(-\log p(x^n\mid \theta ))\right\}.$$

$$p_{\mathrm{NML}}(x^n)\left|\frac{\partial x^n}{\partial y^n}\right|=p_{\mathrm{NML}}(y^n).$$

$$f_P(x)=f_L(x)\sqrt{1+|x{|}^2}{\left(\frac{2}{1-|x{|}^2}\right)}^D,$$

$$-\log f_P(x)+$$

$$=-\log \left(\sqrt{1+|x{|}^2}{\left(\frac{2}{1-|x{|}^2}\right)}^D\right).$$

$$p_{\mathrm{Rm}\text{-}\mathrm{NML}}(x^n)\stackrel{\mathrm{def}}{=}\frac{p_{\mathrm{vol}}(x^n\mid \hat{\theta }(x^n))}{{\int }_{y^n\in {\mathcal{X}}^n}{p}_{\mathrm{vol}}(y^n\mid \hat{\theta }(y^n))d\mathrm{vol}(y^n)}.$$

$$\mathcal{C}(\mathcal{P})={\int }_{y^n\in {\mathcal{X}}^n}{p}_{\mathrm{vol}}(y^n\mid \hat{\theta }(y^n))d\mathrm{vol}(y^n).$$

$${\mathcal{L}}_{\mathrm{Rm}-\mathrm{NML}}\left(x^n\right)\stackrel{\mathrm{def}}{=}-\log p_{\mathrm{Rm}\text{-}\mathrm{NML}}(x^n).$$

$$p_{\mathrm{Rm}\text{-}\mathrm{NML}}(\cdot )$$

$$=\underset{q_{\mathrm{vol}}}{\mathrm{argmin}}\underset{x^n}{\max }\left\{-\log q_{\mathrm{vol}}(x^n)-\underset{\theta }{\min }(-\log p_{\mathrm{vol}}(x^n\mid \theta ))\right\}.$$

$$\underset{x^n}{\max }\left\{-\log q(x^n)-\underset{\theta }{\min }(-\log p(x^n\mid \theta ))\right\}$$

$$=\underset{x^n}{\max }\left\{-\log q_{\mathrm{vol}}(x^n)-\underset{\theta }{\min }(-\log p_{\mathrm{vol}}(x^n\mid \theta ))\right\}.$$

$${\ell }_S=\lceil \underset{x\in S}{\sup }(-{\log }_{2}p(x))-{\log }_2\mathrm{vol}(S)\rceil .$$

Full text

I Introduction

With the recent increase in the scale of graph data, Riemannian data spaces other than Euclidean spaces are attracting attention as latent spaces suitable for graph embedding [52, 24]. For example, hyperbolic spaces have been demonstrated to possess high expressive power for graph data with hierarchical structures [20]. Spherical spaces are particularly effective in representing graph data with cyclic structures [14]. Notably, research on hyperbolic spaces has been particularly remarkable [20]. Specifically, in the field of representation learning, methods that embed hierarchical structures into hyperbolic space have successfully represented such relationships using significantly lower-dimensional space compared to conventional methods based on Euclidean space, while preserving the essential relational information [24]. By assuming a probability distribution over the low-dimensional latent space, which provides a compact and essential representation of data sequences, one can obtain an even more concise and informative characterization of the underlying data structure. When estimating the probabilistic structure behind data, techniques such as model selection and regret analysis are typically employed. Normalized Maximum Likelihood (NML) is known to be a useful approach in these contexts, as it achieves the minimax regret and is widely applied in data analysis based on code-length [37]. Once NML-based encoding is realized, the application of the Minimum Description Length (MDL) principle becomes feasible [49]. For point sequences in Euclidean space, the NML code-length is known to be optimal in terms of minimizing code-length regret [37].

Furthermore, detailed methods for calculating the NML code-length have been extensively studied [26, 29, 42, 43, 48, 32, 30]. These computational methods are theoretically significant as they provide the regret-minimizing code-length. However, while cases where the parameter space forms a Riemannian manifold have been extensively studied in the field of information geometry [3], situations in which the data space itself is a non-Euclidean Riemannian manifold, such as hyperbolic space, pose fundamental challenges to the above framework. There are two non-trivial challenges in extending NML to Riemannian manifolds: how to define it and how to compute it. In what follows, we address both of these issues in detail. We first outline the challenges associated with defining NML on Riemannian manifolds.

For a continuous probability density function $p(x)$, the quantity $-\log p(x)$ is often interpreted as a code-length via a discretization of the space in which $x$ is embedded. However, in the case of data spaces endowed with a Riemannian manifold structure, the discretization process associated with coding is not straightforward. In Euclidean space, for instance, discretization using lattices is possible. On such a discretization, the negative log-likelihood can be interpreted—up to an additive constant determined by the discretization width—as a code-length, particularly when discretizing per unit volume. However, in the case of a Riemannian manifold, no lattice exists that uniformly reflects its intrinsic distance structure. 2. 2.

In Euclidean space, the canonical Cartesian coordinate system exists, and the negative logarithm of a probability density function defined on this system can be interpreted as a code-length. However, on a general Riemannian manifold, no such canonical coordinate system exists. Since probability density functions are coordinate-dependent, interpreting the negative log-density as a code-length introduces ambiguity: the resulting code-length depends on the choice of coordinates and is therefore not uniquely defined. 3. 3.

Without a well-defined notion of code-length, the NML code-length cannot be meaningfully formulated on a general Riemannian manifold.

Even if the above issues were resolved, making such a formulation computationally feasible would remain a significant challenge. In practice, computing the NML code-length is challenging even in Euclidean spaces. The NML code-length is defined as the code-length under the NML distribution, which consists of the maximum likelihood obtained using the maximum likelihood estimator (MLE) of the parameters from the data sequence, and a normalization term over the space of all possible data sequences [37]. This normalization term, known as the parametric complexity (PC), is notoriously difficult to compute. While several methods have been developed to approximate the PC in Euclidean spaces using asymptotic analysis and related techniques [26], it remains non-trivial whether these methods can be extended to Riemannian manifolds, including non-Euclidean spaces.

For Euclidean spaces, several methods for calculating the PC have been proposed in previous studies. Rissanen (1996) derived an asymptotic approximation formula for the PC based on the asymptotic normality of the MLE [26]. Subsequently, Rissanen (2000, 2012) developed a method for calculating the value of PC by partitioning the data sequence space according to the values of the maximum likelihood estimates of the parameters [29, 30]. Mathematical justification of the method for continuous data space cases was recently given [40], based on geometric measure theory. Additionally, Suzuki & Yamanishi (2018, 2021) proposed a Fourier method [42, 43]. Furthermore, Yamanishi (1996) and Roos (2008) proposed a Monte Carlo method for estimating the PC value using sampling approximation [48, 32]. The aforementioned PC calculation methods have been summarized by Yamanishi (2023) [49]. However, most of the existing computational techniques and methodologies have been developed exclusively for probabilistic models defined on Euclidean data spaces. To the best of our knowledge, there exists no prior work that addresses the computation of the NML code-length on Riemannian manifolds.

This study extends the notion of code-length to cases where the data space is a Riemannian manifold and defines the Riemannian manifold Normalized Maximum Likelihood (Rm-NML) code-length, which is optimal in terms of regret with respect to such a code-length. Furthermore, we provide a method for computing the Rm-NML code-length.

The code-length we consider on Riemannian manifolds is defined as the negative logarithm of a probability density function with respect to the base measure induced by the volume element. This corresponds to the code-length obtained by discretizing with unit volume, disregarding multiplicative factors due to discretization methods. To the best of our knowledge, this is the first formulation of its kind. The Rm-NML we define minimizes the regret with respect to this geometrically meaningful notion of code-length, making it the first code-length with such a property.

We also present an asymptotic approximation method for computing the Rm-NML code-length. Specifically, we generalize the method proposed by Rissanen (1996) for Euclidean data spaces to the setting of Riemannian manifolds [26]. Our method avoids integration over the data space’s Cartesian product and provides uniform asymptotic accuracy (with differences of order o(1)) over all data sequences, making it practical for applications. Importantly, the sufficient conditions for applying this method can be verified in any coordinate system, which is crucial because standard coordinate systems generally do not exist on Riemannian manifolds. In fact, we prove that the sufficient conditions are equivalent regardless of the choice of coordinates.

Moreover, we present a simplified computation method for the Rm-NML code-length in cases where the probability density function depends on the Riemannian data point only through the distance function to a parameter. The motivation for considering non-Euclidean spaces, such as hyperbolic space, as data domains lies in the fact that their intrinsic distance structures are well-suited for representing certain types of data. When constructing probabilistic models on such spaces, it is therefore natural to incorporate this underlying geometric structure into the formulation of the probabilistic models. This is a special case of the general method and inherits all its desirable properties. Since most practical applications of Riemannian manifolds focus on their coordinate-invariant distance structure, this simplified computation is widely applicable. As a concrete example, we demonstrate how the Rm-NML code-length can be computed for the hyperbolic Gaussian distribution, a canonical probabilistic model on hyperbolic space.

One of the main contributions of this study is to enable the construction of a code that functions as stochastic complexity (SC) on Riemannian data spaces. SC is understood as a generalization of Shannon information to settings in which the underlying probabilistic model is unknown. When the standard NML is applied to Riemannian data spaces, the Kraft inequality may fail to hold depending on the choice of coordinate system. This issue arises because, on Riemannian manifolds, the standard NML code-length is not correctly defined as a code-length assigned to a partition into unit-volume elements. As a result, this leads to difficulties in applications to information theory. The Rm-NML code-length proposed in this work is coordinate-invariant and is defined with respect to the volume element. Consequently, Rm-NML enables the construction of Huffman codes in a manner that is independent of the choice of coordinate system.

I-A Related research

The NML addressed in this study plays a central role in the MDL principle. The MDL principle is a universal learning strategy grounded in the concept of data compression [27, 30]. Its theoretical validity has been established from various perspectives, consistency as in Rissanen (1998) [28], convergence rate as in Barron and Cover (1991) [5], and sample complexity in stochastic probably approximately correct learning as in Yamanishi (1992) [47]. More detailed theoretical foundations of the MDL principle are discussed in Grünwald (2007) [13]. According to the MDL principle, the best model is the one that minimizes the total description length of the data and the model, as justified from an information-theoretic standpoint. This total description length is characterized by the NML code-length and is established as the SC [30]. The NML code-length is employed because it provides the unique solution to the minimax regret problem, as shown by Shtar’kov [37]. Furthermore, Rissanen (1996) derived an asymptotic approximation for the computation of the NML code-length [26]. One of the representative applications of NML is model selection based on the MDL principle. In this context, the NML code-length serves as a model selection criterion, and the probabilistic model with the shortest NML code-length is selected for a given observed data sequence. Model selection based on the NML code-length has been widely applied, and numerous applications have been reported to date [44, 22, 31, 35, 51, 17, 38, 41, 46, 18, 45, 19]. The MDL principle and NML are comprehensively summarized in Yamanishi (2023) [49].

In recent years, data with non-Euclidean structures has become increasingly common [11, 8], leading to growing interest in Riemannian data spaces [24, 10]. These spaces have garnered particular attention in the context of graph representation learning. For example, hyperbolic spaces have demonstrated remarkable success as embedding spaces for hierarchical graph data [24], while spherical manifolds are known to be well-suited for representing cyclic structures in graphs [52]. Beyond these, other Riemannian manifolds such as tori and Möbius strips have also been explored as data domains [9, 7]. More recently, deep learning has been extended to Riemannian manifolds, enabling geometric learning directly on curved spaces [12]. The theoretical foundation for optimization on Riemannian manifolds has been systematically presented in Absil et al. (2008) [1]. Furthermore, various algorithms have been generalized to Riemannian settings: for instance, Bonnabel (2013) proposed a stochastic gradient descent method on Riemannian manifolds [6], and also extended the expectation-maximization (EM) algorithm to such spaces [33]. Despite this growing body of research, there have been no studies that extend NML—which plays a central role in optimal model selection and regret analysis—to Riemannian data spaces.

As prior research on hyperbolic spaces and the MDL principle, there is a study by Yuki, Ike, and Yamanishi (2021) on dimension selection in hyperbolic spaces [50]. This study employs the Decomposed Normalized Maximum Likelihood Code-length (DNML), a model selection criterion based on the MDL principle for latent variable models [46]. In DNML, the computation of the NML for the observed variables and that for the latent variables are performed separately. This study is the first to adopt non-Euclidean spaces as data spaces and address the MDL principle. Its performance is validated through experiments. However, a key difference from the present study is that Yuki, Ike, and Yamanishi (2021) adopt wrapped normal distributions (WNDs) as the probability model on hyperbolic spaces [23]. WNDs are normal distributions defined on the tangent space of a manifold, analogous to those on Euclidean spaces, and are constructed by mapping the normal distribution onto the manifold via an exponential map. Therefore, the NML is computed according to its standard definition [37], and there are no computational issues. In this study, we consider probability density functions that are defined directly on Riemannian manifolds, without relying on an embedding into or transformation from Euclidean space. As prior work dealing with manifold data spaces and the MDL principle, Yuki, Suzuki & Yamanishi (2023) conducted research on dimension and curvature selection using DNML [46]. This study also adopts WNDs as the probabilistic model.

This study extends the normalized maximum likelihood (NML) code-length so that it can be applied to data spaces that are Riemannian manifolds. In the literature on the MDL principle and NML, there exist studies that interpret the parameter space as a statistical manifold grounded in information geometry. In particular, Balasubramanian (2005) showed that by adopting the Jeffreys prior, which corresponds to a uniform distribution on a statistical manifold from the viewpoint of information geometry, the NML code-length can be derived from a Bayesian perspective and from the geometric structure of parametric model families [4]. This work provides a unified explanation of Occam’s razor, MDL, and Bayesian inference, and demonstrates that the quality of a model is determined not merely by the number of parameters but by its geometric robustness and expressive power. A major difference between that study and the present work lies in the treatment of the data space. While Balasubramanian (2005) views the parameter space as a statistical manifold in the derivation of the general NML, the resulting NML is defined for data in Euclidean spaces. In contrast, the present study proposes Rm-NML, an extension of NML that operates on data spaces endowed with Riemannian manifold structures. As another line of related work that treats the parameter space as a statistical manifold based on information geometry, Sun and Nielsen (2025) evaluated the complexity of deep neural networks under the MDL principle [39]. In their analysis, the parameter space is modeled as a manifold using the framework of singular semi-Riemannian geometry. However, their study also considers data spaces that are Euclidean, which constitutes a key distinction from the present work. As described above, there exist studies on NML that, based on information geometry, regard the parameter space as a Riemannian manifold. In contrast, in this study we treat both the data space and the parameter space as Riemannian manifolds. The proposed Rm-NML is a concept defined on Riemannian-manifold data spaces and yields a quantity that is invariant with respect to the choice of coordinate system on the data space. An important point here is that coordinate transformations in the parameter space and those in the data space are mathematically distinct, and independent coordinate transformations can be applied to each. All concepts introduced in this paper possess coordinate invariance even when such independent transformations are applied. In this way, this study resolves, in a manner consistent with Riemannian geometry, the issue of coordinate invariance in the data space, which has not been addressed in previous work.

I-B Organization

The paper is organized as follows. In Section II, we introduce Riemannian manifolds and the conventional NML. In Section III, we discuss the coordinate dependence of the conventional NML, which poses a major obstacle to its extension to Riemannian manifolds. Section IV presents the definition of Rm-NML, an extension of NML to Riemannian manifolds. In Section V, we summarize the method for computing the PC on Riemannian manifolds, as derived in this study. Section VI presents the computation results of Rm-NML for Gaussian distributions on hyperbolic space, based on the proposed definition and computational method. Finally, Section VII concludes the paper. The appendix contains proofs of theorems and detailed derivations of various results.

II Preliminary

II-A Riemannian manifold

The definition of a Riemannian manifold is given as follows.

Definition 1** (Riemannian manifold).**

Let $\mathcal{M}$ be a differentiable manifold. Suppose that each tangent space $T_{p}\mathcal{M}$ at point $p\in\mathcal{M}$ is equipped with an inner product, given as a positive-definite symmetric bilinear form as follows.

[TABLE]

Let $(U,(x_{1},\cdots,x_{D}))$ be a local coordinate chart around point $p$, and define the following at point $p$.

[TABLE]

If the components $g_{ij}$ are of class $C^{\infty}$ in a neighborhood of each point in $\mathcal{M}$, then the assignment of an inner product $g_{p}$ to each point $p\in\mathcal{M}$ defines a Riemannian metric $g$ on $\mathcal{M}$. In this case, a differentiable manifold equipped with a Riemannian metric is called a Riemannian manifold. As an equivalent definition to Equation (2), the metric tensor can be expressed using the dual basis $\{dx^{1},\cdots,dx^{D}\}$ of the cotangent bundle as follows.

[TABLE]

As a concrete example of a Riemannian manifold, we present the definition of hyperbolic space using the Poincaré ball model.

Definition 2** (Poincaré ball model).**

We define the following space $\mathcal{H}_{p}^{D}$.

[TABLE]

We set $x=(x_{1},\cdots,x_{D})^{\top}$. The $D$-dimensional Poincaré ball model is a space equipped with the following Riemannian metric defined on $\mathcal{H}_{p}^{D}$.

[TABLE]

The Poincaré ball model is equivalent to other hyperbolic models such as the Lorentz model.

Definition 3** (Lorentz model).**

We define the following space $\mathcal{H}_{l}^{D}$.

[TABLE]

[TABLE]

[TABLE]

The $D$-dimensional Lorentz model is a space equipped with the following Riemannian metric defined on $\mathcal{H}_{l}^{D}$.

[TABLE]

A Riemannian manifold $\mathcal{M}$ is a differentiable manifold equipped with a Riemannian metric $g_{\mathcal{M}}(x)$, which assigns an inner product to the tangent space at each point $x\in\mathcal{M}$. The notion of distance on a Riemannian manifold is defined via the Riemannian metric, and integration over the manifold is carried out with respect to a volume element induced by this metric. The definition of the volume element on a Riemannian manifold is provided below.

[TABLE]

Let $x^{n}$ denote a sequence of data points on a Riemannian manifold. The volume element of the n-fold product manifold is defined as follows.

[TABLE]

The Riemannian metric $g_{\mathcal{M}}(x)$ depends on the coordinate system defined on the Riemannian manifold. Consequently, the volume element also becomes a coordinate-dependent quantity. The volume elements defined under two different coordinate systems transform according to the product with the Jacobian determinant between the coordinate systems. From this perspective, functions defined with respect to the volume element are coordinate invariant, since the Jacobian arising from a coordinate transformation is absorbed into the transformation of the volume element itself. Henceforth, probability density functions defined with respect to the volume element will be denoted using the subscript “vol”, such as $p_{\mathrm{vol}}$. The probability density function defined with respect to the volume element and that defined with respect to the coordinate differentials transform as follows.

[TABLE]

As a probability density function defined with respect to the volume element on a Riemannian manifold, the Riemannian Gaussian Distribution (R-GD) has been proposed to generalize the classical normal distribution [34].

Example 1* (R-GD).*

Let $x,\mu\in\mathcal{M},\sigma\in\mathbb{R}$. Let $d(\cdot,\cdot)$ denote the distance between two points induced by the Riemannian metric. On a Riemannian manifold, the probability density function representing a normal distribution, referred to as the Riemannian Gaussian distribution (R-GD), is defined with respect to the volume element as follows.

[TABLE]

[TABLE]

Here, let $D$ denote the dimension of the Riemannian manifold $\mathcal{M}$. Then, the parameter space of R-GD has dimension $D+1$.

II-B Conventional NML

Let $\mathcal{X}$ be the data space, and denote the product set of $n$ copies of $\mathcal{X}$ by $\mathcal{X}^{n}$. Let $\Theta$ be the parameter space, and $\mathcal{P}=\left\{p(x\mid\theta):x\in\mathcal{X},\theta\in\Theta\right\}$ be the probability model class. The NML distribution is defined by normalizing the likelihood specified by the maximum likelihood estimator. Let $x^{n}\in\mathcal{X}^{n}$ be given. The NML distribution is defined as follows [37]. Here, $\hat{\theta}(x^{n})=\operatorname{argmax}_{\theta\in\Theta}p(x^{n}\mid\theta)$ denotes the maximum likelihood estimator.

[TABLE]

The NML distribution defined as above constitutes a valid probability distribution over the sample space $\mathcal{X}^{n}$. The denominator in Equation (15) is referred to as the parametric complexity (PC), which is generally known to be computationally intractable. We denote the PC as follows.

[TABLE]

The NML code-length is defined as the negative logarithm of the NML distribution. Throughout this paper, all logarithms are taken to base 2.

[TABLE]

The NML code-length is established as the SC, and the above expression relates the SC to the PC. The NML code-length achieves the following minimax regret [37]. Let $q$ be a probability density function defined on $\mathcal{X}^{n}$.

[TABLE]

III The coordinate dependence of NML

The definition of conventional NML has been provided above. However, the conventional NML formulation lacks coordinate invariance. When the data sequence $x^{n}$ is transformed into another sequence $y^{n}$, the Jacobian determinant arising from the coordinate transformation appears in the formulation, as shown below.

[TABLE]

Since most previous studies dealing with the MDL principle have been conducted on Euclidean spaces with natural coordinate systems, the lack of coordinate invariance in the NML code-length has not posed a significant issue. However, in the case of Riemannian manifolds, standard coordinate systems do not generally exist, and probability density functions depend on the choice of coordinates. As a result, interpreting the negative logarithm of a probability density as a code-length leads to coordinate-dependent and thus non-unique values. We present below an example in which the probability density function and the code-length vary under a coordinate transformation.

Example 2*.*

Hyperbolic space can be represented by either the Lorentz model or the Poincaré ball model, which are equivalent representations. Suppose a probability density function $f_{L}(x),x\in\mathcal{H}_{l}^{D}$ is defined on the Lorentz model. To express this density in the Poincaré ball model by $f_{P}(x)$, we apply the corresponding coordinate transformation, yielding the following form.

[TABLE]

[TABLE]

Now that the data space has expanded from Euclidean spaces to Riemannian manifolds, it is necessary for the NML code-length to be treated as a coordinate-invariant quantity. In fact, when applying the NML code-length to tasks such as comparing different Riemannian manifolds or detecting changes under settings not aligned with the natural parameterization of Euclidean space, the coordinate dependence can significantly affect performance. In this study, we propose Rm-NML as an extension of the conventional NML framework, designed to operate on Riemannian manifolds while maintaining coordinate invariance. This formulation enables the application of NML-based model selection and code-length analysis in non-Euclidean settings, where standard coordinate systems may not exist and the geometry of the data space plays a crucial role.

IV Riemannian manifold NML

In this study, we propose Rm-NML as an NML that works on Riemannian manifolds with coordinate invariance. Rm-NML is constructed from a probability density function over volume elements, and it is itself a probability density function on volume elements. The definition is as follows.

Definition 4** (Riemannian manifold NML).**

We consider a Riemannian manifold data space $\mathcal{M}$. Let $x^{n}\in\mathcal{X}^{n}$ denote a data sequence defined on the Riemannian manifold $\mathcal{M}$. Let $p_{\mathrm{vol}}(x^{n}\mid\theta)$ be a probability density function on $\mathcal{M}^{n}$, defined with respect to the product volume element induced by the Riemannian structure on $\mathcal{M}$. Let $\hat{\theta}$ be the maximum likelihood estimator of $\theta$. The Rm-NML distribution is then defined as follows.

[TABLE]

By the transformation in Equation (10), it can be seen that the denominator of the above Rm-NML distribution coincides with the PC defined in Equation (16).

[TABLE]

The negative logarithm of the Rm-NML is defined as the Rm-NML code-length.

[TABLE]

The Rm-NML also constitutes a valid probability density function with respect to the volume element on $\mathcal{X}^{n}$. Therefore, the Rm-NML code-length can be interpreted as the information content of a probability density function defined with the volume element as its underlying measure. The Rm-NML achieves the following minimax regret.

[TABLE]

The worst-case regrets in Equations (25) and (18) are equivalent, as indicated by the following relationship.

[TABLE]

The denominator of the Rm-NML coincides with that of the conventional NML, and thus corresponds to the PC. This observation implies that Rm-NML achieves the same worst-case regret as the conventional NML.

IV-A Justification of Rm-NML as a Code-length

The Rm-NML distribution defined in this study is itself a valid probability density function with respect to the Riemannian volume element. By taking its negative logarithm, we obtain the Rm-NML code-length. In this section, we provide a theoretical justification for interpreting the negative log-density—defined with respect to the Riemannian volume element—as a valid code-length.

Consider a continuous space $\mathcal{X}$ and an element $x\in\mathcal{X}$. Consider $\mu$ as a measure. Let $p(x)$ be a probability density function defined on $\mathcal{X}$. Suppose that $\rho=\{S_{1},S_{2},\cdots\}$ is a countable and mutually exclusive partition of $\mathcal{X}$. Under these assumptions, the following two theorems hold.

Theorem 1** (Existence).**

Consider a countable and mutually exclusive partition $\rho$ of the set $\mathcal{X}$. There exists a prefix code for $p(x)$ such that the length $\ell_{S}$ of the codeword assigned to each region $S\in\rho$ satisfies the following condition. Here, let $\operatorname{vol}(S)$ denote the volume of the region S.

[TABLE]

Theorem 2** (Optimality in terms of expected code-length).**

The average code-length $\bar{L}$ of any instantaneous uniquely decodable code with respect to the partition $\rho$ is lower bounded by $L_{\text{lower}}$, which is defined as follows. Here, let $\operatorname{vol}(S)$ denote the volume of the region S.

[TABLE]

When the above two theorems hold and each element of the partition is sufficiently small, both $\sup_{x\in S}p(x)\approx p(x_{S})$ and $\inf_{x\in S}p(x)\approx p(x_{S})$ can be approximated by the value of the density at a representative point $x_{S}\in S$. Under this approximation, the proposed code-length $\ell_{S}$ can be approximated as follows, and it possesses both existence and optimality in terms of the average code-length.

[TABLE]

The definition of information content for a continuous probability density function $-\log p(x)\Delta x=-\log p(x)-\log\Delta x$ is typically derived by considering the information content of the probability $p(x)\Delta x$ over an infinitesimal region $\Delta x$, and neglecting the term associated with $\Delta x$ itself, thereby isolating the component that depends solely on $p(x)$. When considering a partition with sufficiently small elements, the volume $\operatorname{vol}(S)$ becomes infinitesimal. Following the standard definition of information content for continuous probability density functions, this infinitesimal value is ignored in the present study. As a result, a code-length that depends solely on the probability density function, and not on the volume element, can be constructed.

IV-B Relationship with previous studies

As discussed earlier, when applying the MDL principle to data spaces modeled as Riemannian manifolds, it is essential that the resulting code-length possesses coordinate invariance. However, prior studies and previous applications of the MDL principle have largely overlooked the coordinate dependence of the NML formulation. The Rm-NML defined in this study does not refute the existing body of research based on the conventional NML, which lacks coordinate invariance. Rather, by introducing Rm-NML as a coordinate-invariant extension, this work clarifies that the coordinate dependence of conventional NML arises solely from the underlying volume element.

[TABLE]

Most of the previous studies based on the MDL principle have been conducted under the standard setting of Euclidean space. In this setting, the volume element is identically 1 at every point, and thus the Rm-NML coincides with the conventional NML. In other words, the Rm-NML generalizes the standard NML formulation. Furthermore, even when a coordinate system other than the standard Cartesian coordinate system—such as polar coordinates—is employed on Euclidean space, the value of the volume element remains unchanged as long as the data points are fixed. Consequently, the coordinate dependence does not affect model selection performance in such cases.

V Calculation of PC on Riemannian manifolds

The denominator of the Rm-NML is the PC, and its value is derived equivalently from the definition of PC in Euclidean space. In general, PC is known to be computationally intractable, and various techniques have been developed to approximate or compute it. However, once the data space is extended to a Riemannian manifold, it is non-trivial whether existing computational methods remain applicable. In particular, properties that hold only in Euclidean space are no longer valid in this setting. In this study, we extend existing methods for computing PC so that they are applicable on Riemannian data spaces. We verify the validity of these methods under this generalization. Moreover, since the extension to Riemannian manifolds is expected to make the computation of PC even more challenging, we also propose multiple alternative approaches that facilitate the derivation of its value.

V-A Asymptotic Approximation Method

One method for computing the PC is to use an asymptotic approximation, which becomes effective when the number of data points $n$ is sufficiently large [26]. The method for computing the PC using the asymptotic approximation proposed in [26] holds under the following five conditions.

Theorem 3** (Asymptotic approximation formula for PC [26]).**

First, the Fisher information matrix is defined as follows.

[TABLE]

The parameter space $\Theta$ is assumed to be compact. There exist positive constants $c_{1}$ and $c_{2}$ such that for all $\theta\in\Theta$, the inequality $0<c_{1}\leq\lvert I(\theta)\rvert\leq c_{2}<\infty$ holds, where $\lvert I(\theta)\rvert$ denotes the determinant of the Fisher information matrix. 2. 2.

The Fisher information matrix $I(\theta)$ is continuous with respect to $\theta$. 3. 3.

$\int\sqrt{|I(\theta)|}\,d\theta<\infty$. 4. 4.

The asymptotic normality of the maximum likelihood estimator holds uniformly over all points $\theta\in\Theta$. Let $\hat{\theta}(x^{n})$ denote the maximum likelihood estimator of $\theta$ based on the data $x^{n}$.

[TABLE] 5. 5.

There exists a finite positive definite matrix $C_{0}$ such that, for all $n$, and for any $x^{n}$ for which the maximum likelihood estimator $\hat{\theta}(x^{n})$ belongs to $\Theta$, the following inequality holds.

[TABLE]

Under these five conditions, the PC ($\mathcal{C}(\mathcal{P})$) defined in Equation (16) is computed as follows. Here, let $k$ denote the dimension of the parameter space $\Theta$.

[TABLE]

Here, $o(1)$ is taken to be $\lim_{n\to\infty}o(1)=0$.

Remark 1*.*

In this work, we follow Rissanen’s (1996) [26] definition of the Fisher information matrix in terms of the second-order derivatives of the log-likelihood, as given in Equation (31). Alternatively, the Fisher information can also be defined via the first-order derivatives (the score function):

[TABLE]

Let $(\Omega,\mathcal{F},\mathbb{P})$ be a probability space and let $X:\Omega\to\mathcal{X}$ be a random variable with density $f(x;\theta)$. These two definitions coincide under the following conditions:

The data sequence $x_{1},\dots,x_{n}$ is independent and identically distributed (i.i.d.). 2. 2.

For the parameter space $\Theta$, the density $f(x;\theta)$ is of class $C^{2}$ with respect to $\theta$:

[TABLE] 3. 3.

For each $\theta\in\Theta$ and $\mu$-almost every $x\in\mathcal{X}$, we have $f(x;\theta)>0$. 4. 4.

The following integrability conditions hold:

[TABLE]

[TABLE]

where $\mathcal{N}$ is a neighborhood of $\theta$.

When these conditions are satisfied, the log-likelihood for i.i.d. data decomposes as

[TABLE]

and under the framework of Rissanen, the Fisher information matrix based on second-order derivatives coincides with that based on first-order derivatives in the limit of large $n$:

[TABLE]

By Theorem 3 and Equation (17), the NML code-length on a Euclidean data space is given as follows.

Corollary 1** (The NML Code-Length on Euclidean Manifold Data Spaces).**

Under the same conditions as in Theorem 3, the NML code-length on a Euclidean data space can be approximated as follows.

[TABLE]

However, the PC estimation method based on the above asymptotic approximation cannot be directly extended to Riemannian manifolds due to limitations in both its assumptions and formulation.

• •

First, the asymptotic approximation formula for the PC presented in [37] considers a probability density function defined with respect to a density defined by the coordinate system in the Euclidean setting. In contrast, this study extends the data space to a Riemannian manifold and, accordingly, considers probability density functions defined with respect to the volume element induced by the Riemannian metric.

• •

By considering probability density functions defined with respect to the volume element, it becomes necessary to carefully define the Fisher information matrix.

• •

It is straightforward to verify that the asymptotic approximation of the PC provided in [37] is coordinate-invariant. However, it remains unproven whether the conditions required for the validity of the above theorem 3 are preserved under continuous coordinate transformations.

• •

The derivation of the asymptotic approximation relies on quantization of the parameter space using hyperrectangles. Such an approach does not hold in curved spaces, such as Riemannian manifolds, where the notion of hyperrectangles is not directly applicable.

Specifically, we first demonstrate that the Fisher information matrix defined with respect to a probability density function over a density defined by the coordinate system is equivalent to that defined with respect to a probability density function over volume elements. Next, we explicitly specify the coordinate systems of both the data space and the parameter space on the manifold, and prove that the resulting expressions are invariant under smooth coordinate transformations. By making the coordinate systems explicit, the analysis based on hyperrectangles becomes applicable. Furthermore, we show that the assumptions required for the approximation remain valid under this setting. By establishing these two key points, we derive an asymptotic approximation method for the PC on Riemannian manifolds that is independent of the choice of coordinates. In this study, we do not restrict the parameter space to be Euclidean. The proposed formulation is given as follows.

Theorem 4** (Asymptotic approximation formula for PC on Riemannian manifolds).**

Let $p_{\mathrm{vol}}$ be a probability density function defined on the volume element of the Riemannian manifold $\mathcal{M}$. Let $x\in\mathcal{X}\subset\mathcal{M}$ and $\theta\in\Theta$. Set a coordinate system $\phi$ on $\Theta$ to represent $\theta$. Let the Fisher information matrix with respect to coordinate system $\phi$ be $I_{\phi}(\theta)$.

[TABLE]

[TABLE]

The parameter space $\Theta$ is assumed to be compact. There exist positive constants $c_{1}$ and $c_{2}$ such that for all $\theta\in\Theta$, the inequality $0<c_{1}\leq\lvert I_{\phi}(\theta)\rvert\leq c_{2}<\infty$ holds, where $\lvert I_{\phi}(\theta)\rvert$ denotes the determinant of the Fisher information matrix. 2. 2.

The Fisher information matrix $I_{\phi}(\theta)$ is continuous with respect to $\theta$. 3. 3.

$\int_{\theta\in\Theta}\sqrt{|I_{\phi}(\theta)|}\,d\theta<\infty$. 4. 4.

The asymptotic normality of the maximum likelihood estimator holds uniformly over all points $\theta\in\Theta$. Let $\hat{\theta}(x^{n})$ denote the maximum likelihood estimator of $\theta$ based on the data $x^{n}$.

[TABLE] 5. 5.

There exists a finite positive definite matrix $C_{0}$ such that, for all $n$ , and for any $x^{n}$ for which the maximum likelihood estimator $\hat{\theta}(x^{n})$ belongs to $\Theta$, the following inequality holds.

[TABLE]

In this case, the asymptotic approximation formula for PC ($\mathcal{C}(\mathcal{P})$) in Equation (23) can be calculated as follows.

[TABLE]

See Appendix B for the proof.

By Theorem 4 and Equation (24), the Rm-NML code-length on a Riemannian manifold data space is given as follows.

Corollary 2** (The Rm-NML Code-Length on Riemannian Data Spaces).**

Under the same conditions as in Theorem 4, the Rm-NML code-length on a Riemannian Manifold data space can be approximated as follows.

[TABLE]

This expression makes explicit the coordinate system of the parameter $\theta$ in the existing asymptotic approximation. Based on the nontrivial aspects encountered in extending the existing asymptotic approximation method to a Riemannian manifold, we present the derivation of the asymptotic approximation formula for PC on a Riemannian manifold in the form given above. Equation (12) shows that the difference between the probability density functions defined with respect to infinitesimal quantities and those defined with respect to the volume element lies in the volume element itself. Since the volume element is independent of the parameter $\theta$, the Fisher information matrices computed from these two probability density functions are identical.

In conventional asymptotic approximation methods, the parameter space was quantized using hyperrectangles. By introducing a coordinate system $\phi$ on the parameter space $\Theta$, this procedure can be translated into a discussion on Euclidean space.

[TABLE]

Based on the above three points, the existing asymptotic approximation can be extended to a Riemannian manifold data space in the form of Theorem4.

Theorem 4 enables the asymptotic approximation of the value of the PC on a Riemannian manifold in a coordinate-explicit form. However, the Rm-NML distribution defined in this study possesses coordinate invariance, and therefore the computation of the PC, which constitutes the denominator of the distribution, must also be coordinate-invariant. Furthermore, the five regularity conditions assumed in the derivation of the asymptotic approximation must remain valid under continuous coordinate transformations. By demonstrating these two aspects of coordinate invariance, the Rm-NML code-length can be shown to possess coordinate invariance as a model selection criterion.

We first verify the coordinate invariance of the computed result.

Proposition 1* (The asymptotic approximation of the PC on a Riemannian manifold is invariant under coordinate transformations.).*

Let $\phi$ and $\psi$ be coordinate systems related by a continuously differentiable coordinate transformation. The asymptotic approximation of the PC on a Riemannian manifold yields consistent results across these coordinate systems. In particular, the following holds for the second term on the right-hand side of Equation (43).

[TABLE]

Proof.

Let $\phi$ and $\psi$ be coordinate systems related by a continuously differentiable coordinate transformation. Under such a transformation, the determinant of the Fisher information matrix transforms as follows.

[TABLE]

Using Equation (47), we apply the coordinate transformation to the second term on the right-hand side of Equation (43).

[TABLE]

∎

Remark 2*.*

The statement of Proposition 1 is obvious if we regard the determinant $|I_{\phi}(\theta)|$ of the Fisher metric as a volume form. Nevertheless, we have provided the explicit form and a coordinate-based proof for readers’ convenience.

From the result of the above coordinate transformation, it follows that the asymptotic approximation of the PC derived in this study is invariant with respect to the choice of coordinates. In this study, we also prove that the five regularity conditions in Theorem 3 are invariant under coordinate transformations.

Theorem 5** (Coordinate invariance of the regularity conditions).**

Let $\phi$ and $\psi$ be two coordinate systems related by a twice continuously differentiable coordinate transformation. If the five regularity conditions in Theorem 3 hold in the coordinate system $\phi$, then they also hold in the coordinate system $\psi$. See Appendix C for the proof.

In this study, we confirmed that if Conditions 1 through 5 hold in a coordinate system $\phi$ , they also hold in a coordinate system $\psi$ related to $\phi$ via a twice continuously differentiable transformation. Condition 2 is trivial, and the invariance of Condition 3 directly follows from the coordinate invariance of the associated quantity. For Conditions 1 and 5, it can be proven that they are preserved under coordinate transformations based on the transformation of the Fisher information matrix given in Equation (47) and the compactness of the parameter space $\Theta$. Condition 4 is often nontrivial. In this study, we formulated the uniform validity of the asymptotic normality of the maximum likelihood estimator in the form given below, and proved that this condition is preserved under any twice continuously differentiable coordinate transformation111Rissanen (1996), who derived the asymptotic approximation of the PC, did not provide a rigorous definition of the ’uniform validity of the asymptotic normality of the maximum likelihood estimator’ [26]. In this study, we adopt this definition to demonstrate the coordinate invariance of Condition 4; nevertheless, the proof of the asymptotic approximation of the PC given in Rissanen (1996) remains valid under this definition.. Let $\mathcal{X}^{D}$ be a measurable subset of $\mathbb{R}^{D}$ whose boundary has measure zero.

Definition 5** (Uniform validity of the asymptotic normality of the maximum likelihood estimator).**

[TABLE]

Based on the definition of the uniform validity of the asymptotic normality of the maximum likelihood estimator given in Definition 5, the argument presented by Rissanen (1996), who proposed the asymptotic approximation of the PC, is valid [26]. Similarly, the derivation of the asymptotic approximation of the PC on a Riemannian manifold proposed in this study is also justified. The formalization of the uniform validity in Definition 5 expresses ’uniform validity’ explicitly in terms of uniform convergence.

From the above, Theorem 4 possesses coordinate invariance in both its computational result and its regularity conditions. Consequently, the asymptotic approximation given by Theorem 4 enables the calculation of the PC of a probabilistic model as a value intrinsic to the adopted Riemannian manifold data space, unaffected by the choice of coordinate system.

V-B A simple method for computing asymptotic approximations of PC

An asymptotic approximation of the PC on a Riemannian manifold has been obtained. However, the second term on the right-hand side of Equation (43) remains nontrivial and difficult to compute. Among the Riemannian data spaces of interest, spaces with constant curvature—such as hyperbolic spaces and hyperspheres—are of particular relevance. These spaces are instances of Riemannian symmetric spaces. Riemannian symmetric spaces are also Riemannian homogeneous spaces, where isometric transformations act transitively. In this section, we compute the PC for probability density functions defined based on the distance structure of Riemannian homogeneous spaces and Riemannian symmetric spaces. The class of probability density functions considered in this chapter is given as follows.

Definition 6** (Distance-determined-type probability distribution family).**

Let $\mathcal{M}$ be a Riemannian manifold and $d$ be the distance function equipped with $\mathcal{M}$. Also, let $\Gamma$ be a subset of $\mathbb{R}^{m}$, where $m$ is a nonnegative integer. Consider a parametric Riemannian probability density function $p_{\mathrm{vol}}$ on the manifold $\mathcal{M}$ as data space, where $p_{\mathrm{vol}}$ is parametrized by a parameter in the data space $\mathcal{M}$ and another parameter in $\Gamma$. In other words, for any $\theta\in\mathcal{M}$ and $\gamma\in\mathbb{R}^{m}$, $p_{\mathrm{vol}}(\cdot\mid\theta,\gamma):\mathcal{M}\to\mathbb{R}{\geq 0}$ is a Riemannian probability density function on $\mathcal{M}$. Remark that $\theta$ is in the data space $\mathcal{M}$, while $\gamma$ may not. We say that $p_{\mathrm{vol}}$ (or the distribution family represented by $p_{\mathrm{vol}}$) is a distance-determined-type probability distribution family if there exists a function $f:\mathbb{R}{\geq 0}\times\mathbb{R}^{m}\to\mathbb{R}{\geq 0}$ such that

[TABLE]

holds for any $x,\theta\in\mathcal{M}$ and $\gamma\in\Gamma\subset\mathbb{R}^{m}$.

The squared distance is employed in order to circumvent non-differentiable points such as the origin.

R-GD based on Example 1 belongs to this family of distributions. When the parameter $\theta$ represents a representative point of the data, such as the centroid, both $\theta$ and a data point $x$ lie in the same space.

V-B1 Motivation for distance-determined-type Distribution Families

We extend the discussion from Euclidean space to Riemannian manifolds precisely because of our interest in their distance structures; accordingly, the probability density functions should depend on the underlying distance. In this sense, distance-determined-type probability distributions constitute a natural class of distribution families on Riemannian manifolds. If a probability density function does not explicitly depend on distance, then the distance structure of the space may not be fully exploited. This point is particularly important in machine learning applications—most notably in settings that employ hyperbolic space as the data space [24]. Since hyperbolic space is diffeomorphic to Euclidean space, there is little justification for treating the data space as hyperbolic when the probability density function is independent of distance.

V-B2 Examples of distance-determined-type Distribution Families

R-GD belongs to the family of distance-determined-type distributions and can reproduce the Gaussian distribution on an arbitrary Riemannian manifold [34]. In Euclidean space, examples of distance-determined-type distribution families include the Gaussian, Student’s t, Laplace, and Cauchy distributions. On the sphere, the von Mises distribution, which models angular or phase-valued data, is an example of a distance-determined-type distribution family [21]. In hyperbolic space, the hyperbolic Student’s t distribution and the hyperbolic Cauchy distribution are commonly used as examples of distance-determined-type distribution families [15]. In this study, we formulate a theorem that provides a simple method for computing the PC of distance-determined-type distribution families, and this theorem is applicable to all of the distributions mentioned above.

We first define the relevant Fisher information matrices. Let $\phi(\theta)$ be the coordinate system representing the parameter $\theta$ on the Riemannian symmetric space $\mathcal{M}$. We define the combined parameter vector as $\eta=\left(\phi(\theta)^{\top},\gamma^{\top}\right)^{\top}$. The vector $\eta$ is treated as a $D+m$ dimensional vector. The Fisher information matrix of the probability density function $p_{\mathrm{vol}}(x\mid\theta,\gamma)$ is denoted by $I(\phi(\theta),\gamma)$, and is defined as follows.

[TABLE]

[TABLE]

The Fisher information matrix with $\gamma$ fixed is defined as follows.

[TABLE]

[TABLE]

The Fisher information matrix with $\theta$ fixed is defined as follows.

[TABLE]

[TABLE]

In this study, we derive a theorem that simplifies the computation of Equation (43) using the definitions of the Fisher information matrices given above. We define a normal orthonormal basis at a point $\nu$ on a Riemannian manifold $\mathcal{M}$ as a coordinate system in which the Riemannian metric at $\nu$ is represented by the identity matrix.

Theorem 6** (Asymptotic approximation of probability density functions defined by distance and additional parameters on riemannian symmetric spaces).**

We consider a distance-determined-type probability distribution family defined on a Riemannian manifold $\mathcal{M}$. We define the combined parameter vector as $\eta=\left(\phi(\theta)^{\top},\gamma^{\top}\right)^{\top}$. Let $\hat{\eta}=\left(\phi(\hat{\theta})^{\top},\hat{\gamma}^{\top}\right)^{\top}$ denote the maximum likelihood estimator of the parameter. We assume that the five conditions in Theorem 4 also hold with respect to the parameter $\eta$. Based on the above setting, the following results can be established, leading to the derivation of the PC on a Riemannian symmetric space.

For any reference point $\nu\in\mathcal{M}$, we define a coordinate system $\zeta_{\nu}$ on $\nu$ with a normal orthonormal basis. In this setting, fixing $\gamma$ and choosing $\zeta_{\nu}$ as the coordinate system for $\theta$, the determinant of the Fisher information matrix $I_{\zeta_{\nu}}(\theta,\gamma)$ evaluated at $\theta=\nu$, denoted by $|I_{\zeta_{\nu}}(\nu,\gamma)|$, is independent of the choice of $\nu$. We denote this value by $C_{\theta}(\gamma)$.

[TABLE] 2. 2.

Moreover, the determinant $|I_{\gamma}(\theta,\gamma)|$ is independent of $\theta$. We denote this value by $C_{\gamma}(\gamma)$.

[TABLE] 3. 3.

In this case, the asymptotic approximation formula for PC ($\mathcal{C}(\mathcal{P})$) in Equation (23) can be calculated as follows.

[TABLE]

See Appendix D for the proof.

By Theorem 6 and Equation (24), the Rm-NML code-length on a Riemannian manifold data space Under the same conditions as in Theorem 6 is given as follows.

Corollary 3** (The Rm-NML Code-Length on Riemannian Data Spaces).**

Under the same conditions as in Theorem 6, the Rm-NML code-length on a Riemannian Manifold data space can be approximated as follows.

[TABLE]

where $\mathrm{vol}(\Theta)$ denotes the volume of the parameter space $\Theta$.

Remark 3*.*

We remark that the formulae in Theorem 6 have interpretations similar to those in Euclidean space (e.g., [4]. The first term is derived from identifiability and corresponds to the volume of a statistically significant ellipsoid around the estimator $\hat{\theta}$ in the space of probability distributions parameterized by $\theta$, where the Fisher information matrix defines the Riemannian metric. The second and third terms arise from taking the logarithm of the volume of this distribution space. Under the conditions of Theorem 6, Lemmas 3–6 show that this distribution space can be locally represented as a direct product of a parameter space endowed with the same metric as the data space and another space corresponding to the parameter $\gamma$ equipped with a metric induced by the Fisher information. The second term corresponds to the volume of the parameter space with a metric identical to that of the data space, while the third term represents the volume of the $\gamma$-parameter space endowed with the metric derived from the Fisher information. Throughout this analysis, $\mathcal{C}(\mathcal{P})$ represents the proportion of the ellipsoidal neighborhood around $\hat{\theta}$ within the entire distribution space.

Next, we discuss the reasonableness of the Rm-NML results. Corollary 3 provides the resulting expression for the Rm-NML code-length in this setting. One of the central applications of the general NML code-length is model selection. From the viewpoint of statistical learning theory, models with excessively large complexity are undesirable, as they tend to cause overfitting. When the Rm-NML code-length is applied to model selection, the PC derived in Theorem 6 acts as a penalty term for this complexity, namely, the model size. In the result of Corollary 3, the first term is the negative log-likelihood at the maximum likelihood estimator. As the model size increases, the model generally fits the data more easily, and this term therefore decreases. For the Rm-NML to function as a model selection criterion, the subsequent terms must compensate for this decrease by acting as penalties with respect to the model size. In Corollary 3, the second term, $(D+m)$, corresponds to the degrees of freedom of the model and can be regarded as an explicit penalty for the model size. From Theorem B, it follows that the third and fourth terms in Corollary 3 are derived from the geometric volumes of the statistical manifold. In particular, in Corollary 3, the third term is the logarithm of the volume of the parameter space $\Theta$ for $\theta$, and it functions as a penalty term that accounts for the increase in model size with respect to $\theta$. Similarly, the fourth term serves as a penalty for the increase in model size associated with the parameter $\gamma$. From these observations, we conclude that the Rm-NML code-length is consistent with the requirements imposed by statistical learning theory and can serve as a valid model selection criterion.

By applying Theorem 6, the PC for the R-GD on hyperbolic space can be obtained in a straightforward manner, as will be discussed later. Given the above Theorem 6, once the volume of the parameter space is known, the integration over the Riemannian manifold can be avoided.

VI Example: Rm-NML for Gaussian Distributions on Hyperbolic Space

As a concrete example of the proposed Rm-NML, we derive the Rm-NML code-length for the Gaussian distribution defined on hyperbolic space. Hyperbolic space has recently attracted significant attention as an embedding space for hierarchical graph data [20]. This is due to its exponentially expanding geometry, which is well-suited for representing tree-like hierarchical structures. In fact, previous work has demonstrated that while Euclidean embeddings may require up to 200 dimensions to represent certain hierarchical linguistic data, the same structure can be embedded in only 5 dimensions within hyperbolic space [24]. Therefore, hyperbolic space can be regarded as one of the most promising Riemannian manifolds for modeling complex data.

In this study, we focus on the Riemannian Gaussian distribution (R-GD) as a class of probability distributions defined on Riemannian manifolds [34]. The R-GD models probability densities based on the Riemannian distance between a data point and a mean location $\mu$ on the manifold, with a scalar parameter $\sigma$ controlling the dispersion. The R-GD is defined with respect to the volume element on the manifold, making it inherently coordinate-invariant.

Specifically, we consider the Hyperbolic Gaussian Distribution (H-GD), which is an instance of the R-GD defined on hyperbolic space. Since hyperbolic space is a Riemannian symmetric space, and the H-GD belongs to the distance-determined-type probability distribution family defined in this study, we can directly apply Theorem 6. Let $\mathcal{H}^{D}$ denote the hyperbolic space. In this study, we fix the curvature to $-1$. Where $x\in\mathcal{H}^{D}$, $\mu\in\mathcal{H}^{D}$ is the Frèchet mean, $\sigma>0$ is the dispersion parameter, $d_{\mathcal{H}^{D}}(x,\mu)$ denotes the geodesic distance between $x$ and $\mu$, and $\xi_{\mathcal{H}^{D}}(\sigma)$ is the normalizing constant. The H-GD, which is the adaptation of R-GD to hyperbolic space, is then defined as follows.

[TABLE]

[TABLE]

$\xi_{\mathrm{H-GD}}(\sigma)$ is computed as follows. Let $p_{i}$ be defined as $p_{i}=(D-1)-2i$.

[TABLE]

we define $\operatorname{erf}(x)$ as follow.

[TABLE]

The goal of this section is to provide a formula to compute the PC and NML of the H-GD defined above. However, where a distribution family contains a mean parameter or variance parameter as in the (Euclidean) Gaussian distribution and the H-GD, the naive NML does not exist and a modified version of NML has been used. In the following section, we explain the necessary modification, which corresponds to a restriction of data space, clarifying why such restriction is essentially necessary.

VI-A Restriction of the data space

The H-GD has a mean parameter and a variance parameter, as the (Euclidean) Gaussian distribution do. For the Gaussian distribution, it has been known that the NML in the naive sense, characterized as the solution to the minimax regret problem (18), does not exist. The non-existence of the naive NML implies that the original minimax regret problem was overambitious in the sense that the “max” part of the minimax problem considers all the possible data sequences. To relax the naive and infeasible minimax regret problem into a feasible problem, the restriction of the data space in the “max” part of the minmax regret problem has been widely adopted in existing work handling the NML of the (Euclidean) Gaussian distribution, e.g., [29, 17, 16, 43]. Note that the smaller set the “max” part considers, the easier and more feasible a minimax problem. Following the previous work, we also apply the same type of the restriction to formulate the H-GD’s NML. Specifically, we consider the following data space restriction. When restricting the data space, we place the origin $o_{\mathcal{H^{D}}}$ in the hyperbolic space.

[TABLE]

Note that while the above restriction is formally applied to the data space, we can also conceptually view it as a restriction on the parameter space, as the restriction is defined as the location of the maximum likelihood estimator in the parameter space. Under the restriction imposed by Equation (65), the minimax regret problem is modified to the following:

[TABLE]

As the solution to the above modified minimax regret, we obtain the following modified version of Rm-NML.

[TABLE]

In this section, as far as we discuss the H-GD, the Rm-NML refers to the above modified version. Also, the PC refers to the denominator of the modified Rm-NML. Here, we can see that the data space restriction can also be viewed as a restriction in the integral domain in the PC. We can also see that the restriction is consistent to our Theorem 6 in the sense that it leads to satisfy the compactness condition (Condition

1. and the finiteness of the integral (Condition 3). Nevertheless, we emphasize that the necessity of the data space restriction comes from the NML itself, not indicating a flaw of our computation theorems.

Remark 4* (About the choice of R).*

A practical method for selecting the parameter space restriction $R$ is to determine it empirically based on the observed data. In particular, a simple and effective heuristic is to set $R$ equal to the maximum distance from the origin among the observed data points, as suggested in Corollary 5. This approach provides a straightforward way to ensure that the NML code-length is well defined without requiring additional theoretical adjustments. Note that although methods based on renormalized normalized maximum likelihood (RNML) [29, 16] have been proposed to alleviate the dependency of the code-length on the choice of R in the literature of model selection regarding the (Euclidean) Gaussian distribution, extending the RNML to the Riemannian case is nontrivial and remains future work.

VI-B NML computation formula for H-GD

In this subsection, we provide the computation formula for the Rm-NML and PC of the H-GD. We remark that the Rm-NML and PC here are the data space restricted version introduced in the previous subsection. Specifically, the data space is restricted to $\mathcal{X}^{n}(R,\sigma_{\mathrm{min}},\sigma_{\mathrm{max}})$. The following is our asymptotic PC computation formula for H-GD obtained as a corollary of Theorem 6.

Corollary 4** (Computation of the PC for Riemannian Gaussian Distributions on Hyperbolic Space via Asymptotic Analysis).**

When $n$ data points are distributed on the D-dimensional hyperbolic space $\mathcal{H}^{D}$ , the PC of the R-GD can be asymptotically approximated as follows. The data space is restricted to $x^{n}\in\mathcal{X}^{n}(R,\sigma_{\mathrm{min}},\sigma_{\mathrm{max}})$. For notational convenience, we define $B(\sigma)$ as follows.

[TABLE]

Let $\mathrm{V}_{\mathcal{H}^{D}}(R)$ denote the volume of a ball of radius $R$ in D-dimensional hyperbolic space. The PC ($\mathcal{C}(\mathcal{P})$) in Equation (23) for the H-GD on hyperbolic space satisfies the following.

[TABLE]

See Appendix E for the proof.

By Corollary 4, the Rm-NML code-length for the R-GD on hyperbolic space is computed as follows.

Corollary 5** (Computation of the Rm-NML code-length for Riemannian Gaussian Distributions on Hyperbolic Space).**

Under the same conditions as in Corollary 4, the Rm-NML code-length for the R-GD on hyperbolic space is computed as follows.

[TABLE]

Since the integral with respect to $\sigma$ in Corollary 4 and Corollary 5 is analytically intractable, its value is computed using numerical integration.

Each term appearing in Corollary 4 and Corollary 5 admits an interpretation analogous to that given in Remark 3. From the viewpoint of statistical learning theory, models with excessively large complexity are undesirable, as they tend to cause overfitting. One of the central applications of NML is model selection, where the PC is known to function as a penalty term that accounts for the size of the model. With respect to the radius $R$, the model size can be regarded as geometrically monotone increasing. When the dimension $D$ differs, the underlying data domains themselves are different, and hence a direct comparison is not straightforward. Nevertheless, as suggested by the theory underlying criteria such as MDL [29], Akaike Information Criterion (AIC) [2] and Bayesian Information Criterion (BIC) [36], it is generally expected that the model size increases monotonically with respect to the parameter dimension. Equation (69) is monotone increasing in both $D$ and $R$, and is therefore consistent with these requirements arising from statistical learning theory.

The determinant of the Fisher information matrix with respect to the mean parameter $\mu$ becomes a constant due to the coordinate transformation into an orthonormal basis and isometric transformation at each point. Let this constant be denoted as $I_{\zeta_{\nu}}(\nu)$. The resulting expression for the R-GD on hyperbolic space is given below. Here, $\xi^{\prime}(\sigma)$ denotes the derivative of the normalization factor $\xi(\sigma)$ with respect to $\sigma$, and $E$ is the D-dimensional identity matrix.

[TABLE]

[TABLE]

The Fisher information $I_{\sigma}$ with respect to the scale parameter $\sigma$ for the R-GD on hyperbolic space is given as follows.

[TABLE]

$\mathrm{V}_{\mathcal{H}^{D}}(R)$ denote the volume of a ball of radius $R$ in D-dimensional hyperbolic space. This volume is given by the following expression [25].

[TABLE]

In Theorem 6, it is shown that for distance-determined-type families on Riemannian symmetric spaces, the PC can be computed once the following quantities are known: the volume of the parameter space of parameters that lie on the same Riemannian manifold as the data points, the determinant of the Fisher information matrix with respect to those parameters, and the determinant of the Fisher information matrix with respect to the remaining parameters. In the example of the hyperbolic Gaussian distribution, the integration range of the parameter space is restricted to the region where the distance from the origin is less than or equal to $R$. Therefore, if the volume of a ball in hyperbolic space is known, the volume of the parameter space can be computed. Since a closed-form formula for this volume exists, the calculation is straightforward. Moreover, in this example, Theorem 6 implies that the determinant of the Fisher information matrix can be computed in a factorized manner. As a result, it is not necessary to evaluate mixed derivative terms involving both $\theta$ and $\sigma$, which significantly simplifies the derivation of the determinant. Corollary 5 is a concrete example of Theorem 6, and it supports the claim that the computation is indeed simplified by Theorem 6. Moreover, by computing the parametric complexity, we can quantify how much additional code-length is incurred when encoding R-GD on hyperbolic space using Rm-NML code-length.

VII Conclusion

In this study, we have presented, for the first time, a theoretical framework for calculating the NML code-length on Riemannian data spaces in a coordinate-invariant manner. The proposed Rm-NML formulation allows for the computation of code-lengths that are invariant under coordinate transformations, thereby ensuring geometric consistency in statistical modeling on curved spaces.

By employing Rm-NML, it becomes possible to conduct regret analysis based on the MDL principle on Riemannian manifolds, just as has been done using NML in Euclidean spaces, while preserving geometric consistency. The computation of the PC, however, is generally challenging, and it was previously unclear whether asymptotic approximation techniques—originally developed in Euclidean settings—could be extended to Riemannian data spaces. In this study, we derive asymptotic approximations of PC on Riemannian manifolds and demonstrate that both the approximated expressions and their regularity conditions are preserved under smooth coordinate transformations. This provides a coordinate-invariant method for computing the asymptotic form of PC on Riemannian data spaces. Nevertheless, even with such approximations, computing PC remains difficult due to the extension of the data space to general Riemannian manifolds.

To address this, we focus on data spaces that exhibit symmetric structures, which have received increasing attention in recent studies. Specifically, we develop a simplified approach to computing the PC for probability distributions defined based on the geodesic structure of Riemannian symmetric spaces. This method allows us to avoid integration over the data space when the volume of the parameter space is known, thereby significantly reducing the computational complexity of PC. Furthermore, this framework is applicable to Riemannian manifolds that have attracted substantial interest in recent years. As a concrete example, we provide the asymptotic form of the PC for R-GD defined on hyperbolic space.

By utilizing the Rm-NML, it becomes possible to apply the MDL principle on data spaces endowed with Riemannian geometry, while preserving geometric consistency. As a result, various applications of regret-based analysis using NML can now be extended to Riemannian manifolds.

Appendix A Information Theoretic Justification of Code-Lengths for Continuous Probability Distributions

In this section, we give the proofs of Theorem 1 and Theorem 2, which establish an information-theoretic justification for code-lengths in continuous probability distributions. We first prove the existence result stated in Theorem 1.

Proof.

For a prefix code with code-lengths $\ell_{S}$ to exist, the Kraft inequality $\sum_{S\in\rho}2^{-\ell_{S}}\leq 1$ must be satisfied. Let us verify this condition. By definition, we have $\ell_{S}\geq\sup_{x\in S}\bigl(-\log_{2}p(x)\bigr)-\log_{2}\operatorname{vol}(S)$.

[TABLE]

Transform the right-hand side.

[TABLE]

Therefore, we can conclude the following.

[TABLE]

Here, since $\inf_{x\in S}(p(x))\cdot\operatorname{vol}(S)\leq\int_{S}p(x)d\mu(x)$, the following holds.

[TABLE]

Since $p(x)$ is a probability density function, $\int_{\mathcal{X}}p(x)\,d\mu(x)=1$, and hence $\sum_{s\in\rho}2^{-\ell_{s}}\leq 1$ holds. This implies the existence of a prefix code with codeword lengths $\ell_{s}$. ∎

Next, we show that the coding scheme described in the previous section is optimal in terms of the average code-length. Let the probability that $x$ belongs to a region $S$ be given by

[TABLE]

Therefore, the average code-length required to specify the region $S$ to which $x$ belongs is written as

[TABLE]

where $\ell^{*}_{S}$ denotes the codeword length assigned to region $S$.

Proof.

By the source coding theorem, the average code-length of any instantaneous decodable code (including prefix codes) cannot be smaller than the entropy. That is,

[TABLE]

We now show that the average code-length is bounded from below by $L_{\mathrm{lower}}$.

[TABLE]

To this end, we consider the difference between $H(\{P(S)\})$ and $L_{\mathrm{lower}}$:

[TABLE]

Since

[TABLE]

it follows that

[TABLE]

and hence each term in the sum is nonnegative. Therefore,

[TABLE]

Consequently,

[TABLE]

which shows that the average code-length cannot be smaller than $\mathbb{E}_{S}\!\left[\inf_{x\in S}(-\log_{2}p(x))-\log_{2}\operatorname{vol}(S)\right]$. ∎

By establishing these two theorems, the Rm-NML defined in Definition 4 can be interpreted as a valid code-length.

Appendix B Asymptotic approximation formula for PC on Riemannian manifolds

We provide a proof of the asymptotic approximation of the PC on a Riemannian manifold, which is derived in this study and presented as Theorem 4.

Proof.

In Theorem 3, the parameter is denoted by $\theta$, while in Theorem 4, it is denoted by $\phi(\theta)$. This distinction in notation is used to emphasize the claim of coordinate invariance. Even in Theorem 3, integration and differentiation are carried out with respect to the specified coordinate system. Theorem 3 is proven, following Rissanen (1996), using a hyper-rectangular partition of the parameter space. Although such a construction cannot be explicitly employed when dealing with Riemannian manifolds, the second term on the right-hand side of the asymptotic approximation in Theorem 4 involves integration with respect to the coordinate representation of the parameter, rather than with respect to the volume element. Therefore, the proof technique based on hyper-rectangles remains valid. The key issue lies in the definition of the Fisher information matrix.

The definitions of the Fisher information matrix in Theorem 3 and Theorem 4 yield the same numerical values. As previously discussed, there is no issue regarding differentiation. The key distinction lies in the definition of the probability density function: in Theorem 3, it is defined with respect to infinitesimal coordinate volume elements, whereas in Theorem 4, it is defined with respect to the Riemannian volume element. Nevertheless, as shown in the following derivation, the resulting Fisher information matrices are equivalent.

[TABLE]

As demonstrated above, the Fisher information matrices yield identical values. Therefore, Theorem 4 holds in the same manner as Theorem 3. ∎

Appendix C Coordinate Invariance of the Conditions for the Asymptotic Approximation

We prove that the five regularity conditions stated in Theorem 4 are preserved under any twice continuously differentiable coordinate transformation. Conditions 2, which concerns the continuity of the Fisher information matrix, and 3, which requires the finiteness of the resulting PC value, are considered to be self-evident under coordinate transformations. Here, we assume that all conditions are satisfied with respect to a coordinate system $\phi$, and consider a transformation to another coordinate system $\psi$ via a twice continuously differentiable change of coordinates.

C-A Coordinate Invariance of Condition 1

Assume that $\Theta$ is compact, and that there exist positive constants $c_{1},c_{2}$ such that for all $\theta\in\Theta$, $0<c_{1}\leq|I_{\phi}(\theta)|\leq c_{2}<\infty$. Let $\psi$ be the coordinate system obtained from $\phi$ via a twice continuously differentiable transformation. Then, we aim to prove that $\Theta$ remains compact, and that there exist positive constants $c_{1}^{\prime},c_{2}^{\prime}$ such that for all $\theta\in\Theta$, $0<c_{1}^{\prime}\leq|I_{\psi}(\theta)|\leq c_{2}^{\prime}<\infty$.

Proof.

[TABLE]

Since $\det\frac{\partial\psi}{\partial\phi}(\theta)$ is a continuous function with respect to $\theta$, and $\Theta$ is compact, there exists a maximum value $b_{max}$. From Condition 4, since $I_{\psi}(\theta)<C_{0}^{\prime}$, $I_{\psi}$ is bounded and does not diverge. Moreover, from Condition 1, $\det\frac{\partial\psi}{\partial\phi}(\theta)$ is non-zero.

[TABLE]

[TABLE]

[TABLE]

From the above, it follows that $c_{2}^{\prime}=\frac{c1}{b_{max}^{2}}$, $c_{2}^{\prime}=\frac{c2}{b_{min}^{2}}$, and hence $c_{1}^{\prime},c_{2}^{\prime}$ exist. ∎

C-B Coordinate Invariance of Condition 5

To verify whether the following condition remains valid even under coordinate transformations that are twice continuously differentiable, we consider coordinate systems obtained via such transformations. Suppose there exists a finite positive definite matrix $C_{0}$ such that the following inequality holds:

[TABLE]

Let $\psi$ be a coordinate system obtained by applying a twice continuously differentiable coordinate transformation to the coordinate system $\phi$. Suppose there exists a finite positive definite matrix $C_{0}^{\prime}$ such that the following also holds:

[TABLE]

Proof.

The Jacobian matrix

[TABLE]

is a matrix-valued function that is continuous with respect to $\theta$, and the parameter space $\Theta$ is compact. Therefore, the matrix norm attains its maximum value:

[TABLE]

Then, the Fisher information matrix under the transformed coordinate system $\psi$ can be written as:

[TABLE]

Using the inequality in Equation (86) and the assumption on $I_{\phi}(x^{n},\theta)$, we obtain:

[TABLE]

Therefore, letting

[TABLE]

we conclude that such a $C_{0}^{\prime}$ exists. ∎

C-C Coordinate Invariance of Condition 4

We verify that the uniform asymptotic normality as defined in Definition 5 is preserved under twice continuously differentiable coordinate transformations. Since the proof of the coordinate invariance of Condition 4 is quite lengthy, we present only an outline of the proof.

We begin by establishing the following two lemmas.

Lemma 1*.*

Suppose the function $\nabla_{\phi}\log p(x^{n}\mid\theta)$ is twice continuously differentiable with respect to $\phi(\theta)$, and that the maximum likelihood estimator $\hat{\theta}$ satisfies

[TABLE]

If the following asymptotic normality holds for some $\theta_{0}\in\Theta$,

[TABLE]

then we have

[TABLE]

Lemma 2*.*

Assume that $\nabla_{\phi}\log p(x^{n}\mid\phi(\theta))$ is twice continuously differentiable with respect to $\phi(\theta)$, and that the maximum likelihood estimator $\hat{\theta}=\hat{\theta}(x^{n})$ satisfies

[TABLE]

When the regularity conditions for asymptotic normality hold, the following result also holds for the reparameterized coordinate system $\psi(\theta)$, which is obtained by a smooth and invertible transformation $\psi$ of the parameter $\theta$:

[TABLE]

Both of the above results can be derived using the first-order Taylor expansion, the law of large numbers, and Slutsky’s theorem. Using these lemmas, we establish the coordinate invariance of Condition 4. The term bounded by $\epsilon$ in Definition 5 is decomposed as follows, and by considering an open ball whose radius increases with $n$, the desired result is obtained.

[TABLE]

[TABLE]

Appendix D A simple method for computing asymptotic approximations of PC

To prove Theorem 6, we establish the following four lemmas.

Lemma 3*.*

Under the same setting as Theorem 6, let $\mathcal{M}$ be a Riemannian manifold and $\theta\in\mathcal{M}$ be an arbitrary point. The coordinate transformation of the determinant of the Fisher information matrix $I_{\phi}(\theta,\gamma)$, defined with respect to a global coordinate system $\phi$ and a locally orthonormal coordinate system $\zeta^{\theta}$ at $\theta$, satisfies the following expression.

[TABLE]

Lemma 4*.*

Under the same setting as Theorem 6, consider a coordinate system $\zeta^{\nu}$ defined by an orthonormal basis at an arbitrary reference point $\nu\in\mathcal{M}$ on the Riemannian manifold $\mathcal{M}$. Then, the determinant of the Fisher information matrix $I_{\zeta^{\nu}}(\nu,\gamma)$, evaluated at $\nu$ for fixed $\gamma$, is independent of the choice of $\nu$.

[TABLE]

Lemma 5*.*

Under the same setting as Theorem 6, the Fisher information matrix $I(\phi(\theta),\gamma)$ becomes block-diagonal and can be expressed as follows:

[TABLE]

Lemma 6*.*

Under the same setting as Theorem 6, the Fisher information matrix $I_{\gamma}(\nu,\gamma)$, with respect to $\gamma$, is independent of $\theta$, and satisfies:

[TABLE]

Lemmas 3, 5, and 6 can be proven by relatively straightforward variable transformations. In contrast, the proof of Lemma 4 relies on the use of isometric transformations. Theorem 6 considers a Riemannian symmetric space, which is also a Riemannian homogeneous space. In a Riemannian homogeneous space, isometries exist transitively; that is, for any two points, there exists an isometry that maps one to the other. An isometry is a mapping that preserves distances. Lemma 4 is proven by considering orthonormal bases at two points that are related via such an isometry, and by computing the determinant of the Fisher information matrix at each point.

Appendix E Computation of the PC for H-GD via Asymptotic Analysis

We derive the PC value of R-GD on hyperbolic space. By applying Theorem 6, the integral over hyperbolic space can be avoided. Thus, the quantities that actually need to be evaluated are the volume of the parameter space $\Theta$ and the constants $C_{\theta}(\gamma)$ and $C_{\gamma}(\gamma)$ appearing in Theorem 6. The volume of the parameter space can be computed as the volume of a geodesic ball in hyperbolic space, restricted to a radius $R$, and is given by Equation (74). There exist multiple equivalent models for representing hyperbolic space. In this work, we adopt the Lorentz model with the following metric and proceed with the calculations accordingly.

[TABLE]

The polar coordinate representation in the Lorentz model is given as follows.

[TABLE]

Theorem 6 asserts that the value of $C_{\theta}(\gamma)$ is constant and does not depend on the choice of representative point $\nu$. As the representative point, we consider the origin $o_{h}$ in the Lorentz model, where the coordinate index satisfies $(\phi_{0},\phi_{1},\cdots,\phi_{D})=(1,0,\cdots,0)$. As a result of the computation, the following expression is obtained.

[TABLE]

[TABLE]

Due to symmetry, it is sufficient to compute a single diagonal component of $I_{\phi}\left(o_{h},\sigma\right)$. In the following, we consider the (1,1)-component and denote the diagonal entry by $I_{\phi}\left(o_{h},\sigma\right)_{i,i}$. By rewriting Equation (101) in terms of the polar coordinate representation, the constant $I_{\phi}\left(o_{h},\sigma\right)$ can be decomposed as follows.

[TABLE]

We now evaluate the two integrals presented above, respectively. For convenience, we set $p_{i}=D-1-2i$ and $C(D,i)=\binom{D-1}{i}(-1)^{i}\exp\left(\frac{\sigma^{2}}{2}p_{i}^{2}\right)\left\{\operatorname{erf}\left(\frac{\sigma p_{i}}{\sqrt{2}}\right)+1\right\}$.

[TABLE]

[TABLE]

The derivation of Equation (104) utilizes the Wallis integral.

The derivative of the normalization term of R-GD $\xi(\sigma)$ on hyperbolic space with respect to $\sigma$ is given as follows.

[TABLE]

Using Equation (105), the constant $I_{\phi}\left(o_{h},\sigma\right)_{i,i}$ can be expressed in a simplified form as follows.

[TABLE]

From the above results, it follows that $I_{\phi}\left(o_{h},\sigma\right)$ is a diagonal matrix whose diagonal entries are given by the value in Equation (106).

We now proceed to evaluate $I_{\sigma}(\sigma)$. We begin by computing the second derivative of the log-likelihood function $L$ with respect to $\sigma$.

[TABLE]

Based on the above computation, the Fisher information $I_{\sigma}(\sigma)$ is calculated as follows.

[TABLE]