Direct From Darwin: Deriving Advanced Optimizers From Evolutionary First Principles

TL;DR

This paper derives advanced gradient-based optimization algorithms from evolutionary principles, showing that popular methods like SGD and Adam can be made evolutionarily faithful through structured noise, bridging optimization and Darwinian evolution.

Contribution

It introduces Darwinian Lineage Simulations to connect evolutionary theory with modern optimization algorithms, enabling evolutionarily faithful simulations of methods like SGD, Newton's methods, and Adam.

Findings

Fisher's and Wright's evolution views are formally equivalent in an asexual context.

Adding structured noise (DLS noise) makes optimization algorithms evolutionarily faithful.

State-of-the-art Adam optimizer can be adapted to evolutionary principles.

Abstract

Evolutionary computation has long promised to deliver both high-performance optimization tools as well as rigorous scientific simulations of Darwinian evolution. However, modern algorithms frequently abandon evolutionary fidelity for physics-inspired heuristics or superficial biological metaphors. This paper derives a suite of advanced gradient-based optimization algorithms directly from evolutionary first principles. We introduce Darwinian Lineage Simulations (DLS) to prove that, in an asexual context, Fisher's and Wright's historically opposed views of evolution are actually formally equivalent; One can partition Fisher's deterministically-evolving total population into Wright's randomly-drifting sub-populations. We prove that proper bookkeeping requires introducing a specific kind of structured noise (the DLS noise relation). Crucially, any bookkeeping choices which satisfy this relation will yield a faithful simulation of evolution. Using this vast representational freedom, we prove that a broad family of battle-tested optimization algorithms are already perfectly compatible with evolutionary dynamics. These include: Stochastic Gradient Descent as well as many regularizations/approximations of Newton's method and Natural Gradient Descent. By simply adding DLS noise (i.e., evolutionarily faithful genetic drift), these algorithms become scientifically valid in silico simulations of Darwinian evolution. Finally, we demonstrate that even the state-of-the-art Adam optimizer can be brought into evolutionary compliance through a minor mathematical surgery.