A systolic update scheme to overcome memory bandwidth limitations in GPU-accelerated FDTD simulations

TL;DR

This paper presents a systolic update scheme for 3D FDTD simulations on GPUs, significantly improving performance by reducing memory bandwidth limitations, thus enabling faster photonic simulations for AI applications.

Contribution

The paper introduces a novel systolic update scheme for FDTD that minimizes global memory access and synchronization, enhancing GPU-based photonic simulation efficiency.

Findings

Achieves 0.15 trillion cell updates per second on Nvidia H100 GPU.

Reduces global memory access to boundary values only.

Improves simulation throughput for photonic design workflows.

Abstract

The exponential growth of artificial intelligence has fueled the development of high-bandwidth photonic interconnect fabrics as a critical component of modern AI supercomputers. As the demand for ever-increasing AI compute and connectivity continues to grow, the need for high-throughput photonic simulation engines to accelerate and even revolutionize photonic design and verification workflows will become an increasingly indispensable capability for the integrated photonics industry. Unfortunately, the mainstay and workhorse of photonic simulation algorithms, the finite-difference time-domain (FDTD) method, because it is a memory-intensive but computationally-lightweight algorithm, is fundamentally misaligned with modern computational platforms which are equipped to deal with compute intensive workloads instead. This paper introduces a systolic update scheme for the FDTD method, which circumvents this mismatch by reducing the need for global synchronization while also relegating the need to access global memory to the case of boundary values between neighboring subdomains only. We demonstrate a practical implementation of our scheme as applied to the full three-dimensional FDTD algorithm that achieves a performance of roughly 0.15 trillion cell updates per second (TCUPS) on a single Nvidia H100 GPU. Our work paves the way for the increasingly efficient, cost-effective, and high-throughput photonic simulation engines needed to continue powering the AI era.