Symmetry Breaking in Neuroevolution: A Technical Report

TL;DR

This paper investigates symmetry breaking in neuroevolution, introduces an adaptive strategy based on the global optimum proximity principle, and demonstrates its effectiveness in improving global search efficiency across various neural network problems.

Contribution

It proposes a novel, adaptive symmetry breaking strategy applicable to different evolutionary algorithms, addressing conflicting reports on symmetry breaking efficacy.

Findings

Significant improvements in global search efficiency with the proposed method.

The strategy's effectiveness varies depending on the problem's symmetry properties.

Application to DE and CMA-ES shows broad applicability.

Abstract

Artificial Neural Networks (ANN) comprise important symmetry properties, which can influence the performance of Monte Carlo methods in Neuroevolution. The problem of the symmetries is also known as the competing conventions problem or simply as the permutation problem. In the literature, symmetries are mainly addressed in Genetic Algoritm based approaches. However, investigations in this direction based on other Evolutionary Algorithms (EA) are rare or missing. Furthermore, there are different and contradictionary reports on the efficacy of symmetry breaking. By using a novel viewpoint, we offer a possible explanation for this issue. As a result, we show that a strategy which is invariant to the global optimum can only be successfull on certain problems, whereas it must fail to improve the global convergence on others. We introduce the \emph{Minimum Global Optimum Proximity} principle as a generalized and adaptive strategy to symmetry breaking, which depends on the location of the global optimum. We apply the proposed principle to Differential Evolution (DE) and Covariance Matrix Adaptation Evolution Strategies (CMA-ES), which are two popular and conceptually different global optimization methods. Using a wide range of feedforward ANN problems, we experimentally illustrate significant improvements in the global search efficiency by the proposed symmetry breaking technique.