Optimization of hybrid parallel application execution in heterogeneous high performance computing systems considering execution time and power consumption

TL;DR

This paper develops a general model and optimization methodology for hybrid parallel applications in heterogeneous HPC systems, aiming to minimize execution time and power consumption amidst complex interdependencies.

Contribution

It synthesizes existing approaches into a unified model and proposes an optimization method considering both execution time and power use in heterogeneous HPC systems.

Findings

Proposed a general model for hybrid parallel application execution.

Developed an optimization methodology balancing time and power.

Analyzed the impact of process mapping on performance and energy.

Abstract

Many important computational problems require utilization of high performance computing (HPC) systems that consist of multi-level structures combining higher and higher numbers of devices with various characteristics. Utilizing full power of such systems requires programming parallel applications that are hybrid in two meanings: they can utilize parallelism on multiple levels at the same time and combine together programming interfaces specific for various types of computing devices. The main goal of parallel processing is increasing the processing performance, and therefore decreasing the application execution time. The international HPC community is targeting development of "Exascale" supercomputers (able to sustain $10^{18}$ floating point operations per second) by the year 2020. One of the main obstacles to achieving this goal is power consumption of the computing systems that exceeds the energy supply limits. New programming models and algorithms that consider this criterion are one of the key areas where significant progress is necessary in order to achieve the goal. The goal of the dissertation is to extract a general model of hybrid parallel application execution in heterogeneous HPC systems that is a synthesis of existing specific approaches and developing an optimization methodology for such execution aiming for minimization of the contradicting objectives of application execution time and power consumption of the utilized computing hardware. Both meanings of the application hybridity result in multiplicity of execution parameters of nontrivial interdependences and influence on the considered optimization criteria. Mapping of the application processes on computing devices has also a significant impact on these criteria.