Universality in Numerical Computations with Random Data. Case Studies

TL;DR

This paper demonstrates that the fluctuations in the convergence time of various numerical algorithms with random data exhibit a universal pattern, independent of input data distribution, as the problem size grows.

Contribution

It provides empirical evidence for universality in halting time fluctuations across multiple algorithms and models, extending the concept to neural computation and decision making.

Findings

Halting time fluctuations follow a universal distribution after normalization.

Universality holds across six numerical algorithms and neural models.

The results suggest a fundamental principle governing stochastic numerical computations.

Abstract

The authors present evidence for universality in numerical computations with random data. Given a (possibly stochastic) numerical algorithm with random input data, the time (or number of iterations) to convergence (within a given tolerance) is a random variable, called the halting time. Two-component universality is observed for the fluctuations of the halting time, i.e., the histogram for the halting times, centered by the sample average and scaled by the sample variance, collapses to a universal curve, independent of the input data distribution, as the dimension increases. Thus, up to two components, the sample average and the sample variance, the statistics for the halting time are universally prescribed. The case studies include six standard numerical algorithms, as well as a model of neural computation and decision making. A link to relevant software is provided in for the reader who would like to do computations of his'r own.