Pruning as Evolution: Emergent Sparsity Through Selection Dynamics in Neural Networks

TL;DR

This paper introduces an evolutionary framework for neural network pruning, modeling parameters as populations subject to selection dynamics, enabling sparsity emergence without explicit pruning schedules and maintaining competitive accuracy.

Contribution

It formalizes neural pruning as an evolutionary process with selection dynamics, connecting fitness to local learning signals, and demonstrates its effectiveness on MNIST.

Findings

Pruning based on evolutionary dynamics achieves high accuracy at significant sparsity levels.

The framework reaches near baseline accuracy (~98%) with 35-50% sparsity.

Evolutionary pruning produces a measurable accuracy-sparsity tradeoff without explicit schedules.

Abstract

Neural networks are commonly trained in highly overparameterized regimes, yet empirical evidence consistently shows that many parameters become redundant during learning. Most existing pruning approaches impose sparsity through explicit intervention, such as importance-based thresholding or regularization penalties, implicitly treating pruning as a centralized decision applied to a trained model. This assumption is misaligned with the decentralized, stochastic, and path-dependent character of gradient-based training. We propose an evolutionary perspective on pruning: parameter groups (neurons, filters, heads) are modeled as populations whose influence evolves continuously under selection pressure. Under this view, pruning corresponds to population extinction: components with persistently low fitness gradually lose influence and can be removed without discrete pruning schedules and without requiring equilibrium computation. We formalize neural pruning as an evolutionary process over population masses, derive selection dynamics governing mass evolution, and connect fitness to local learning signals. We validate the framework on MNIST using a population-scaled MLP (784--512--256--10) with 768 prunable neuron populations. All dynamics reach dense baselines near 98\% test accuracy. We benchmark post-training hard pruning at target sparsity levels (35--50\%): pruning 35\% yields $\approx$95.5\% test accuracy, while pruning 50\% yields $\approx$88.3--88.6\%, depending on the dynamic. These results demonstrate that evolutionary selection produces a measurable accuracy--sparsity tradeoff without explicit pruning schedules during training.