Hyperellipsoid Density Sampling: Exploitative Sequences to Accelerate High-Dimensional Optimization

TL;DR

Hyperellipsoid Density Sampling (HDS) is an adaptive, non-uniform sampling method that accelerates high-dimensional optimization by focusing on promising regions, outperforming traditional quasi-Monte Carlo methods in benchmark tests.

Contribution

This paper introduces HDS, a novel adaptive sampling strategy using hyperellipsoids and unsupervised learning to improve high-dimensional optimization efficiency.

Findings

HDS significantly outperforms Sobol in solution accuracy.

HDS achieves up to 37% performance improvement in 10D.

HDS is versatile for applications requiring focused sampling.

Abstract

The curse of dimensionality presents a pervasive challenge in optimization problems, with exponential expansion of the search space rapidly causing traditional algorithms to become inefficient or infeasible. An adaptive sampling strategy is presented to accelerate optimization in this domain as an alternative to uniform quasi-Monte Carlo (QMC) methods. This method, referred to as Hyperellipsoid Density Sampling (HDS), generates its sequences by defining multiple hyperellipsoids throughout the search space. HDS uses three types of unsupervised learning algorithms to circumvent high-dimensional geometric calculations, producing an intelligent, non-uniform sample sequence that exploits statistically promising regions of the parameter space and improves final solution quality in high-dimensional optimization problems. A key feature of the method is optional Gaussian weights, which may be provided to influence the sample distribution towards known locations of interest. This capability makes HDS versatile for applications beyond optimization, providing a focused, denser sample distribution where models need to concentrate their efforts on specific, non-uniform regions of the parameter space. The method was evaluated against Sobol, a standard QMC method, using differential evolution (DE) on the 29 CEC2017 benchmark test functions. The results show statistically significant improvements in solution geometric mean error (p < 0.05), with average performance gains ranging from 3% in 30D to 37% in 10D. This paper demonstrates the efficacy of HDS as a robust alternative to QMC sampling for high-dimensional optimization.