NEVO-GSPT: Population-Based Neural Network Evolution Using Inflate and Deflate Operators

TL;DR

This paper presents NEVO-GSPT, a neural network evolution method using geometric semantic operators and a novel size reduction operator, enabling efficient search for compact, high-performing architectures with reduced computational cost.

Contribution

It introduces NGS-PT, combining geometric semantic operators with size control, to improve neural architecture search efficiency and understanding of structural impacts.

Findings

Achieves comparable or better performance than existing methods.

Evolves compact neural networks efficiently.

Reduces computational cost of neural architecture search.

Abstract

Evolving neural network architectures is a computationally demanding process. Traditional methods often require an extensive search through large architectural spaces and offer limited understanding of how structural modifications influence model behavior. This paper introduces \gls{ngspt}, a novel Neuroevolution algorithm based on two key innovations. First, we adapt geometric semantic operators~(GSOs) from genetic programming to neural network evolution, ensuring that architectural changes produce predictable effects on network semantics within a unimodal error surface. Second, we introduce a novel operator (DGSM) that enables controlled reduction of network size, while maintaining the semantic properties of~GSOs. Unlike traditional approaches, \gls{ngspt}'s efficient evaluation mechanism, which only requires computing the semantics of newly added components, allows for efficient population-based training, resulting in a comprehensive exploration of the search space at a fraction of the computational cost. Experimental results on four regression benchmarks show that \gls{ngspt} consistently evolves compact neural networks that achieve performance comparable to or better than established methods in the literature, such as standard neural networks, SLIM-GSGP, TensorNEAT, and SLM.