The Mating Rituals of Deep Neural Networks: Learning Compact Feature Representations through Sexual Evolutionary Synthesis

TL;DR

This paper introduces sexual evolutionary synthesis for deep neural networks, combining two parent networks to produce more diverse and efficient offspring, leading to more compact feature representations without sacrificing accuracy.

Contribution

It proposes a novel sexual evolutionary synthesis method for neural networks, enhancing architectural efficiency and diversity over traditional asexual methods.

Findings

Improved architectural efficiency on MNIST and CIFAR-10 datasets.

Achieved approximately double the efficiency compared to asexual synthesis.

Maintained comparable testing accuracy (~97%) with more compact networks.

Abstract

Evolutionary deep intelligence was recently proposed as a method for achieving highly efficient deep neural network architectures over successive generations. Drawing inspiration from nature, we propose the incorporation of sexual evolutionary synthesis. Rather than the current asexual synthesis of networks, we aim to produce more compact feature representations by synthesizing more diverse and generalizable offspring networks in subsequent generations via the combination of two parent networks. Experimental results were obtained using the MNIST and CIFAR-10 datasets, and showed improved architectural efficiency and comparable testing accuracy relative to the baseline asexual evolutionary neural networks. In particular, the network synthesized via sexual evolutionary synthesis for MNIST had approximately double the architectural efficiency (cluster efficiency of 34.29X and synaptic efficiency of 258.37X) in comparison to the network synthesized via asexual evolutionary synthesis, with both networks achieving a testing accuracy of ~97%.