The Impact of Space-Filling Curves on Data Movement in Parallel Systems

TL;DR

This paper investigates how space-filling curves like Hilbert and Morton can enhance data locality and movement efficiency in parallel scientific applications, especially stencil-based computations, on modern hierarchical memory systems.

Contribution

It provides empirical analysis of space-filling curves' impact on data movement and locality in parallel systems, focusing on stencil applications and hardware interactions.

Findings

Space-filling curves improve data locality for certain application parameters.

Performance benefits depend on hardware characteristics and application specifics.

Space-filling curves influence data sharing patterns in stencil computations.

Abstract

Modern computer systems are characterized by deep memory hierarchies, composed of main memory, multiple layers of cache, and other specialized types of memory. In parallel and distributed systems, additional memory layers are added to this hierarchy. Achieving good performance for computational science applications, in terms of execution time, depends on the efficient use of this diverse and hierarchical memory. This paper revisits the use of space-filling curves to specify the ordering in memory of data structures used in representative scientific applications executing on parallel machines containing clusters of multicore CPUs with attached GPUs. This work examines the hypothesis that space-filling curves, such as Hilbert and Morton ordering, can improve data locality and hence result in more efficient data movement than row or column-based orderings. First, performance results are presented that show for what application parameterizations and machine characteristics this is the case, and are interpreted in terms of how an application interacts with the computer hardware and low-level software. This research particularly focuses on the use of stencil-based applications that form the basis of many scientific computations. Second, how space-filling curves impact data sharing in nearest-neighbour and stencil-based codes is considered.