Analytical derivatives of Neural Networks

TL;DR

This paper introduces a recursive algorithm for efficiently computing first- and second-order derivatives of deep neural networks, applicable to quantum mechanics problems like modeling ground-state wave functions.

Contribution

It presents a novel recursive method for derivatives in neural networks, integrating parameter derivatives, and demonstrates its application to quantum variational problems.

Findings

Effective computation of derivatives for arbitrary DFNNs

Application to quantum mechanical ground-state modeling

Potential for improved neural network optimization

Abstract

We propose a simple recursive algorithm that allows the computation of the first- and second-order derivatives with respect to the inputs of an arbitrary deep feed forward neural network (DFNN). The algorithm naturally incorporates the derivatives with respect to the network parameters. To test the algorithm, we apply it to the study of the quantum mechanical variational problem for few cases of simple potentials, modeling the ground-state wave function in terms of a DFNN.