Criticality analysis of nuclear binding energy neural networks

TL;DR

This paper introduces a non-empirical criticality analysis method based on renormalization group flows to understand and optimize neural networks used for nuclear physics, specifically for nuclear binding energy predictions.

Contribution

It adapts a criticality analysis framework to neural networks in nuclear physics, providing insights into their internal behavior and optimization beyond empirical tuning.

Findings

Criticality behavior aligns with training using stochastic gradient descent.

Optimal network performance occurs at critical tuning points.

Adaptive learning algorithms reduce the need for critical tuning, affecting analysis.

Abstract

Machine learning methods, in particular deep learning methods such as artificial neural networks (ANNs) with many layers, have become widespread and useful tools in nuclear physics. However, these ANNs are typically treated as ``black boxes'', with their architecture (width, depth, and weight/bias initialization) and the training algorithm and parameters chosen empirically by optimizing learning based on limited exploration. We test a non-empirical approach to understanding and optimizing nuclear physics ANNs by adapting a criticality analysis based on renormalization group flows in terms of the hyperparameters for weight/bias initialization, training rates, and the ratio of depth to width. This treatment utilizes the statistical properties of neural network initialization to find a generating functional for network outputs at any layer, allowing for a path integral formulation of the ANN outputs as a Euclidean statistical field theory. We use a prototypical example to test the applicability of this approach: a simple ANN for nuclear binding energies. We find that with training using a stochastic gradient descent optimizer, the predicted criticality behavior is realized, and optimal performance is found with critical tuning. However, the use of an adaptive learning algorithm leads to somewhat superior results without concern for tuning and thus obscures the analysis. Nevertheless, the criticality analysis offers a way to look within the black box of ANNs, which is a first step towards potential improvements in network performance beyond using adaptive optimizers.