Towards Co-execution on Commodity Heterogeneous Systems: Optimizations for Time-Constrained Scenarios

TL;DR

This paper presents optimizations for the EngineCL runtime to improve co-execution performance on commodity heterogeneous systems, focusing on reducing overheads and enhancing load balancing under time constraints.

Contribution

It introduces specific optimizations and an improved load balancing algorithm that significantly enhance performance and efficiency in heterogeneous system co-execution scenarios.

Findings

Optimization improvements of 7.5% and 17.4% for different offloading modes.

The new load balancing algorithm achieves an average efficiency of 0.84.

Optimizations reduce performance penalties in time-constrained co-execution.

Abstract

Heterogeneous systems are present from powerful supercomputers, to mobile devices, including desktop computers, thanks to their excellent performance and energy consumption. The ubiquity of these architectures in both desktop systems and medium-sized service servers allow enough variability to exploit a wide range of problems, such as multimedia workloads, video encoding, image filtering and inference in machine learning. Due to the heterogeneity, some efforts have been done to reduce the programming effort and preserve performance portability, but these systems include a set of challenges. The context in which applications offload the workload along with the management overheads introduced when doing co-execution, penalize the performance gains under time-constrained scenarios. Therefore, this paper proposes optimizations for the EngineCL runtime to reduce the penalization when co-executing in commodity systems, as well as algorithmic improvements when load balancing. An exhaustive experimental evaluation is performed, showing optimization improvements of 7.5\% and 17.4\% for binary and ROI-based offloading modes, respectively. Thanks to all the optimizations, the new load balancing algorithm is always the most efficient scheduling configuration, achieving an average efficiency of 0.84 under a pessimistic scenario.