# Hessian-information geometric formulation of a class of deterministic neural network models

## TL;DR

This paper introduces a geometric framework for deterministic neural network models using Hessian and information geometry, linking phase space properties to differential operators on manifolds, with explicit calculations for sigmoid activations.

## Contribution

It formulates neural network dynamics within a Hessian geometric framework, connecting phase space compressibility to Laplace operators on Hessian manifolds, and explicitly analyzes sigmoid functions.

## Key findings

- Phase space compressibility expressed via Laplace operator on Hessian manifolds.
- Explicit calculation of compressibility for sigmoid activation functions.
- Utilization of dual coordinates in information geometry for neural network analysis.

## Abstract

In this paper a class of dynamical systems describing deterministic neural network models are formulated from a viewpoint of differential geometry. This class includes the Hopfield model and gradient systems, and is such that the so-called activation functions induce information and Hessian geometries. In this formulation, it is shown that the phase space compressibility of a dynamical system belonging to this class is written in terms of the Laplace operator defined on Hessian manifolds, where phase space compressibility is associated with a volume-form of a manifold, and expresses how such a volume-form is compressed along the vector field of a dynamical system. Since the sigmoid function, as an activation function, plays a role in the study of neural network models, such compressibility is explicitly calculated for this case. Throughout this paper, the so-called dual coordinates known in information geometry are explicitly used.