Finding Strong Lottery Ticket Networks with Genetic Algorithms

TL;DR

This paper introduces a genetic algorithm-based method to identify strong lottery ticket sub-networks in neural networks without training, outperforming gradient-based methods on small binary classification tasks.

Contribution

It is the first to use genetic algorithms for finding lottery tickets, demonstrating superior results on certain tasks compared to existing gradient-based approaches.

Findings

Genetic algorithms can effectively find lottery tickets without training.

The approach outperforms gradient-based methods on small binary classification tasks.

Smaller, better-performing lottery ticket networks are discovered using evolutionary strategies.

Abstract

According to the Strong Lottery Ticket Hypothesis, every sufficiently large neural network with randomly initialized weights contains a sub-network which - still with its random weights - already performs as well for a given task as the trained super-network. We present the first approach based on a genetic algorithm to find such strong lottery ticket sub-networks without training or otherwise computing any gradient. We show that, for smaller instances of binary classification tasks, our evolutionary approach even produces smaller and better-performing lottery ticket networks than the state-of-the-art approach using gradient information.