MMES: Mixture Model based Evolution Strategy for Large-Scale Optimization

TL;DR

This paper introduces MMES, a scalable evolution strategy for large-scale optimization that uses a mixture model for sampling, maintaining efficiency and effectiveness compared to existing methods.

Contribution

The paper proposes a novel mixture model sampling method and mutation strength adaptation for CMA-ES, enabling scalable large-scale optimization with preserved invariance.

Findings

Reduces time complexity of CMA-ES significantly

Maintains rotational invariance in high dimensions

Performs competitively against state-of-the-art methods

Abstract

This work provides an efficient sampling method for the covariance matrix adaptation evolution strategy (CMA-ES) in large-scale settings. In contract to the Gaussian sampling in CMA-ES, the proposed method generates mutation vectors from a mixture model, which facilitates exploiting the rich variable correlations of the problem landscape within a limited time budget. We analyze the probability distribution of this mixture model and show that it approximates the Gaussian distribution of CMA-ES with a controllable accuracy. We use this sampling method, coupled with a novel method for mutation strength adaptation, to formulate the mixture model based evolution strategy (MMES) -- a CMA-ES variant for large-scale optimization. The numerical simulations show that, while significantly reducing the time complexity of CMA-ES, MMES preserves the rotational invariance, is scalable to high dimensional problems, and is competitive against the state-of-the-arts in performing global optimization.