A pencil distributed finite difference code for strongly turbulent wall-bounded flows

TL;DR

This paper introduces a high-performance, pencil domain decomposition-based finite difference code optimized for simulating strongly turbulent wall-bounded flows, with excellent scalability and reduced communication overhead.

Contribution

The paper presents a novel numerical scheme and parallelization strategy that significantly reduces communication and improves scalability for turbulent flow simulations.

Findings

Achieves very good strong and weak scaling up to 64K cores.

Reduces all-to-all communication to only six instances.

Utilizes implicit time integration to avoid non-local communication at high CFL conditions.

Abstract

We present a numerical scheme geared for high performance computation of wall-bounded turbulent flows. The number of all-to-all communications is decreased to only six instances by using a two-dimensional (pencil) domain decomposition and utilizing the favourable scaling of the CFL time-step constraint as compared to the diffusive time-step constraint. As the CFL condition is more restrictive at high driving, implicit time integration of the viscous terms in the wall-parallel directions is no longer required. This avoids the communication of non-local information to a process for the computation of implicit derivatives in these directions. We explain in detail the numerical scheme used for the integration of the equations, and the underlying parallelization. The code is shown to have very good strong and weak scaling to at least 64K cores.