Unwinding the model manifold: choosing similarity measures to remove local minima in sloppy dynamical systems

TL;DR

This paper uses information geometry to analyze parameter sensitivities in complex dynamical systems, proposing a method to choose similarity measures that simplify the model manifold and reduce local minima issues.

Contribution

It introduces a classification scheme for model sensitivities, defines a curvature measure called winding frequency, and shows how alternative metrics can simplify the model manifold structure.

Findings

Sloppy models exhibit exponential hierarchies of parameter sensitivities.

High effective-dimensionality models have complex, multimodal fitting landscapes.

Alternative metrics can transform the model manifold into simpler structures like hyper-ribbons.

Abstract

In this paper, we consider the problem of parameter sensitivity in models of complex dynamical systems through the lens of information geometry. We calculate the sensitivity of model behavior to variations in parameters. In most cases, models are sloppy, that is, exhibit an exponential hierarchy of parameter sensitivities. We propose a parameter classification scheme based on how the sensitivities scale at long observation times. We show that for oscillatory models, either with a limit cycle or a strange attractor, sensitivities can become arbitrarily large, which implies a high effective-dimensionality on the model manifold. Sloppy models with a single fixed point have model manifolds with low effective-dimensionality, previously described as a "hyper-ribbon". In contrast, models with high effective dimensionality translate into multimodal fitting problems. We define a measure of curvature on the model manifold which we call the \emph{winding frequency} that estimates the linear density of local minima in the model's parameter space. We then show how alternative choices of fitting metrics can "unwind" the model manifold and give low winding frequencies. This prescription translates the model manifold from one of high effective-dimensionality into the "hyper-ribbon" structures observed elsewhere. This translation opens the door for applications of sloppy model analysis and model reduction methods developed for models with low effective-dimensionality.