# Extreme Scale FMM-Accelerated Boundary Integral Equation Solver for Wave Scattering

## TL;DR

This paper presents an optimized, large-scale FMM-accelerated boundary integral equation solver for wave scattering, demonstrating high efficiency and scalability on exascale HPC architectures with significant performance gains.

## Contribution

It introduces a highly optimized, architecture-aware FMM-based solver for wave scattering that scales efficiently on exascale supercomputers, utilizing advanced parallelism and SIMD optimizations.

## Key findings

- Achieves 77% of peak performance on Skylake processors.
- Demonstrates near-linear weak scalability up to 6,144 nodes.
- Computes over 2 billion degrees of freedom on Cray XC40.

## Abstract

Algorithmic and architecture-oriented optimizations are essential for achieving performance worthy of anticipated energy-austere exascale systems. In this paper, we present an extreme scale FMM-accelerated boundary integral equation solver for wave scattering, which uses FMM as a matrix-vector multiplication inside the GMRES iterative method. Our FMM Helmholtz kernels treat nontrivial singular and near-field integration points. We implement highly optimized kernels for both shared and distributed memory, targeting emerging Intel extreme performance HPC architectures. We extract the potential thread- and data-level parallelism of the key Helmholtz kernels of FMM. Our application code is well optimized to exploit the AVX-512 SIMD units of Intel Skylake and Knights Landing architectures. We provide different performance models for tuning the task-based tree traversal implementation of FMM, and develop optimal architecture-specific and algorithm aware partitioning, load balancing, and communication reducing mechanisms to scale up to 6,144 compute nodes of a Cray XC40 with 196,608 hardware cores. With shared memory optimizations, we achieve roughly 77% of peak single precision floating point performance of a 56-core Skylake processor, and on average 60% of peak single precision floating point performance of a 72-core KNL. These numbers represent nearly 5.4x and 10x speedup on Skylake and KNL, respectively, compared to the baseline scalar code. With distributed memory optimizations, on the other hand, we report near-optimal efficiency in the weak scalability study with respect to both the logarithmic communication complexity as well as the theoretical scaling complexity of FMM. In addition, we exhibit up to 85% efficiency in strong scaling. We compute in excess of 2 billion DoF on the full-scale of the Cray XC40 supercomputer.