The sloppy model universality class and the Vandermonde matrix

TL;DR

This paper explores why many complex models are 'sloppy', showing they often share universal spectral features and proposing a Vandermonde matrix ensemble as a universal class for such models.

Contribution

It introduces the Vandermonde ensemble as a universal model class for sloppy systems and demonstrates their shared spectral properties.

Findings

Eigenvalue spectra of sloppy models have a flat density of log-eigenvalues.

Sloppy models exhibit a universal spectral form across different fields.

Vandermonde ensemble captures the universal features of sloppy models.

Abstract

In a variety of contexts, physicists study complex, nonlinear models with many unknown or tunable parameters to explain experimental data. We explain why such systems so often are sloppy; the system behavior depends only on a few `stiff' combinations of the parameters and is unchanged as other `sloppy' parameter combinations vary by orders of magnitude. We contrast examples of sloppy models (from systems biology, variational quantum Monte Carlo, and common data fitting) with systems which are not sloppy (multidimensional linear regression, random matrix ensembles). We observe that the eigenvalue spectra for the sensitivity of sloppy models have a striking, characteristic form, with a density of logarithms of eigenvalues which is roughly constant over a large range. We suggest that the common features of sloppy models indicate that they may belong to a common universality class. In particular, we motivate focusing on a Vandermonde ensemble of multiparameter nonlinear models and show in one limit that they exhibit the universal features of sloppy models.