Scaling Behaviors of Evolutionary Algorithms on GPUs: When Does Parallelism Pay Off?

TL;DR

This study systematically investigates how GPU parallelism affects the behavior and performance of various evolutionary algorithms across different problem sizes and settings, revealing when parallelism is beneficial or not.

Contribution

It provides a comprehensive empirical analysis of 16 EAs on GPUs versus CPUs, highlighting the importance of evaluation methods and problem characteristics in understanding GPU benefits.

Findings

GPU impact varies with algorithmic structure and problem size.

Fixed-time evaluation reveals performance traits hidden by fixed-budget metrics.

Large populations on GPUs enhance hardware utilization and expose unique convergence behaviors.

Abstract

Evolutionary algorithms (EAs) are increasingly implemented on graphics processing units (GPUs) to leverage parallel processing capabilities for enhanced efficiency. However, existing studies largely emphasize the raw speedup obtained by porting individual algorithms from CPUs to GPUs. Consequently, these studies offer limited insight into when and why GPU parallelism fundamentally benefits EAs. To address this gap, we investigate how GPU parallelism alters the behavior of EAs beyond simple acceleration metrics. We conduct a systematic empirical study of 16 representative EAs on 30 benchmark problems. Specifically, we compare CPU and GPU executions across a wide range of problem dimensionalities and population sizes. Our results reveal that the impact of GPU acceleration is highly heterogeneous and depends strongly on algorithmic structure. We further demonstrate that conventional fixed-budget evaluation based on the number of function evaluations (FEs) is inadequate for GPU execution. In contrast, fixed-time evaluation uncovers performance characteristics that are unobservable under small or practically constrained FE budgets, particularly for adaptive and exploration-oriented algorithms. Moreover, we identify distinct scaling regimes in which GPU parallelism is beneficial, saturates, or degrades as problem dimensionality and population size increase. Crucially, we show that large populations enabled by GPUs not only improve hardware utilization but also reveal algorithm-specific convergence and diversity dynamics that are difficult to observe under CPU-constrained settings. Consequently, our findings indicate that GPU parallelism is not strictly an implementation detail, but a pivotal factor that influences how EAs should be evaluated, compared, and designed for modern computing platforms.