Load Balancing For High Performance Computing Using Quantum Annealing

TL;DR

This paper explores the use of quantum annealing for load balancing in high performance computing, demonstrating its advantages in complex particle simulations and comparing its performance to classical methods.

Contribution

It introduces a general quantum annealing-based load balancing protocol applicable to different HPC algorithms and shows its superior performance in multi-objective optimization scenarios.

Findings

Quantum annealing outperforms classical methods in complex particle simulations.

In grid-based applications, quantum annealing is competitive but not decisively better.

Limited hardware coupling currently restricts scalability of quantum annealing.

Abstract

With the advent of exascale computing, effective load balancing in massively parallel software applications is critically important for leveraging the full potential of high performance computing systems. Load balancing is the distribution of computational work between available processors. Here, we investigate the application of quantum annealing to load balance two paradigmatic algorithms in high performance computing. Namely, adaptive mesh refinement and smoothed particle hydrodynamics are chosen as representative grid and off-grid target applications. While the methodology for obtaining real simulation data to partition is application specific, the proposed balancing protocol itself remains completely general. In a grid based context, quantum annealing is found to outperform classical methods such as the round robin protocol but lacks a decisive advantage over more advanced methods such as steepest descent or simulated annealing despite remaining competitive. The primary obstacle to scalability is found to be limited coupling on current quantum annealing hardware. However, for the more complex particle formulation, approached as a multi-objective optimization, quantum annealing solutions are demonstrably Pareto dominant to state of the art classical methods across both objectives. This signals a noteworthy advancement in solution quality which can have a large impact on effective CPU usage.