Embedded Ensemble Propagation for Improving Performance, Portability and Scalability of Uncertainty Quantification on Emerging Computational Architectures

TL;DR

This paper introduces embedded ensemble propagation, a novel method that enhances uncertainty quantification simulations by leveraging modern architectures for better performance, scalability, and portability across diverse computing platforms.

Contribution

The paper presents a new embedded ensemble propagation technique that reuses data across samples, improving efficiency and scalability in uncertainty quantification on emerging architectures.

Findings

Significant performance improvements on CPU, GPU, and accelerators.

Effective scalability up to 131,072 cores on a Cray XK7.

Enhanced portability across different hardware architectures.

Abstract

Quantifying simulation uncertainties is a critical component of rigorous predictive simulation. A key component of this is forward propagation of uncertainties in simulation input data to output quantities of interest. Typical approaches involve repeated sampling of the simulation over the uncertain input data, and can require numerous samples when accurately propagating uncertainties from large numbers of sources. Often simulation processes from sample to sample are similar and much of the data generated from each sample evaluation could be reused. We explore a new method for implementing sampling methods that simultaneously propagates groups of samples together in an embedded fashion, which we call embedded ensemble propagation. We show how this approach takes advantage of properties of modern computer architectures to improve performance by enabling reuse between samples, reducing memory bandwidth requirements, improving memory access patterns, improving opportunities for fine-grained parallelization, and reducing communication costs. We describe a software technique for implementing embedded ensemble propagation based on the use of C++ templates and describe its integration with various scientific computing libraries within Trilinos. We demonstrate improved performance, portability and scalability for the approach applied to the simulation of partial differential equations on a variety of CPU, GPU, and accelerator architectures, including up to 131,072 cores on a Cray XK7 (Titan).