Evolving Neural Networks with Optimal Balance between Information Flow and Connections Cost

TL;DR

This paper explores evolving neural network architectures by balancing connection costs and information flow, using network science principles to improve accuracy across multiple datasets.

Contribution

It introduces a novel approach that incorporates network science metrics to optimize neural network architecture evolution, focusing on the balance between connections cost and information flow.

Findings

Improved accuracy on three datasets.

Metrics based on network science enhance evolution process.

Balancing connection cost and information flow benefits neural network performance.

Abstract

Evolving Neural Networks (NNs) has recently seen an increasing interest as an alternative path that might be more successful. It has many advantages compared to other approaches, such as learning the architecture of the NNs. However, the extremely large search space and the existence of many complex interacting parts still represent a major obstacle. Many criteria were recently investigated to help guide the algorithm and to cut down the large search space. Recently there has been growing research bringing insights from network science to improve the design of NNs. In this paper, we investigate evolving NNs architectures that have one of the most fundamental characteristics of real-world networks, namely the optimal balance between connections cost and information flow. The performance of different metrics that represent this balance is evaluated and the improvement in the accuracy of putting more selection pressure toward this balance is demonstrated on three datasets.