An Application of the Holonomic Gradient Method to the Neural Tangent Kernel

TL;DR

This paper introduces a numerical method based on holonomic systems and computer algebra to evaluate dual activations in neural tangent kernels, enhancing computational techniques in neural network analysis.

Contribution

It applies the holonomic gradient method to neural tangent kernels, providing a novel computational approach for evaluating holonomic distributions in neural networks.

Findings

Developed algorithms for numerical evaluation of dual activations.

Applied computer algebra to holonomic systems in neural tangent kernels.

Enhanced computational efficiency in neural network analysis.

Abstract

A holonomic system of linear partial differential equations is, roughly speaking, a system whose solution space is finite dimensional. A distribution that is a solution of a holonomic system is called a holonomic distribution. We give methods to numerically evaluate dual activations of holonomic activator distributions for neural tangent kernels. These methods are based on computer algebra algorithms for rings of differential operators.