Optimization and Supervised Machine Learning Methods for Fitting Numerical Physics Models without Derivatives

TL;DR

This paper explores optimization and supervised learning techniques for calibrating complex, computationally expensive nuclear physics models without derivative information, focusing on hyperparameter tuning and stochastic optimization in limited evaluation scenarios.

Contribution

It introduces methods for effective model calibration without derivatives, emphasizing hyperparameter tuning and stochastic optimization strategies for costly models.

Findings

Hyperparameter tuning significantly impacts optimization performance.

Stochastic algorithms exhibit variability that affects calibration accuracy.

Limited model evaluations require careful optimization strategy selection.

Abstract

We address the calibration of a computationally expensive nuclear physics model for which derivative information with respect to the fit parameters is not readily available. Of particular interest is the performance of optimization-based training algorithms when dozens, rather than millions or more, of training data are available and when the expense of the model places limitations on the number of concurrent model evaluations that can be performed. As a case study, we consider the Fayans energy density functional model, which has characteristics similar to many model fitting and calibration problems in nuclear physics. We analyze hyperparameter tuning considerations and variability associated with stochastic optimization algorithms and illustrate considerations for tuning in different computational settings.