Multiform Evolution for High-Dimensional Problems with Low Effective Dimensionality

TL;DR

This paper introduces a multiform evolutionary algorithm that leverages multiple low-dimensional embeddings to efficiently solve high-dimensional problems with low effective dimensionality, improving convergence speed.

Contribution

It presents a novel multiform optimization framework that unifies low-dimensional search tasks into a multi-task setting for high-dimensional problems.

Findings

Effective in high-dimensional benchmark functions

Accelerates hyper-parameter tuning in machine learning

Improves convergence in deep learning and game scenarios

Abstract

In this paper, we scale evolutionary algorithms to high-dimensional optimization problems that deceptively possess a low effective dimensionality (certain dimensions do not significantly affect the objective function). To this end, an instantiation of the multiform optimization paradigm is presented, where multiple low-dimensional counterparts of a target high-dimensional task are generated via random embeddings. Since the exact relationship between the auxiliary (low-dimensional) tasks and the target is a priori unknown, a multiform evolutionary algorithm is developed for unifying all formulations into a single multi-task setting. The resultant joint optimization enables the target task to efficiently reuse solutions evolved across various low-dimensional searches via cross-form genetic transfers, hence speeding up overall convergence characteristics. To validate the overall efficacy of our proposed algorithmic framework, comprehensive experimental studies are carried out on well-known continuous benchmark functions as well as a set of practical problems in the hyper-parameter tuning of machine learning models and deep learning models in classification tasks and Predator-Prey games, respectively.