Equilibrium Propagation and Hamiltonian Inference in the Diffusive Fitzhugh-Nagumo Model

TL;DR

This paper extends equilibrium propagation to skew-gradient systems, demonstrating equivalences between deep energy-based models and Hamiltonian neural networks using Fitzhugh-Nagumo neuron networks.

Contribution

It introduces a framework connecting equilibrium propagation with Hamiltonian inference in diffusively coupled Fitzhugh-Nagumo models, revealing new theoretical links.

Findings

Stationary solutions described by self-adjoint operators enable credit assignment methods.

Deep residual Fitzhugh-Nagumo networks admit a spatial Hamiltonian for steady states.

Derived explicit layer-wise Hamiltonian recurrence relation for inference.

Abstract

In this work, we extend the Equilibrium Propagation framework to skew-gradient systems and show an equivalence between deep Energy-Based Models and Hamiltonian neural networks. We focus on networks of diffusively coupled Fitzhugh-Nagumo neurons as a prototypical example. We show that since stationary solutions of the Fitzhugh-Nagumo model are described by self-adjoint operators, the methods of equilibrium propagation for performing credit assignment can be applied. Furthermore, for Fitzhugh-Nagumo networks with the topology of a deep residual network, we show that the steady state solutions admit a (spatial) Hamiltonian, and thus the methods of Hamiltonian Echo Backpropagation can be applied. We end by deriving an explicit layer-wise Hamiltonian recurrence relation governing inference for stationary solutions of both deep Fitzhugh-Nagumo networks and deep Energy-Based Models.