TaPS: A Performance Evaluation Suite for Task-based Execution Frameworks
J. Gregory Pauloski, Valerie Hayot-Sasson, Maxime Gonthier, Nathaniel, Hudson, Haochen Pan, Sicheng Zhou, Ian Foster, Kyle Chard

TL;DR
TaPS is a comprehensive evaluation suite designed to standardize performance assessment of task-based execution frameworks across various applications and hardware, facilitating fair comparison and research advancement.
Contribution
It introduces a modular interface and reference applications for benchmarking task-based execution frameworks, addressing the lack of evaluation standards.
Findings
TaPS enables consistent performance evaluation across frameworks.
It supports synthetic and real-world application benchmarking.
Demonstrated through a survey of benchmark results.
Abstract
Task-based execution frameworks, such as parallel programming libraries, computational workflow systems, and function-as-a-service platforms, enable the composition of distinct tasks into a single, unified application designed to achieve a computational goal. Task-based execution frameworks abstract the parallel execution of an application's tasks on arbitrary hardware. Research into these task executors has accelerated as computational sciences increasingly need to take advantage of parallel compute and/or heterogeneous hardware. However, the lack of evaluation standards makes it challenging to compare and contrast novel systems against existing implementations. Here, we introduce TaPS, the Task Performance Suite, to support continued research in parallel task executor frameworks. TaPS provides (1) a unified, modular interface for writing and evaluating applications using arbitrary…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsCloud Computing and Resource Management · Software System Performance and Reliability · Real-Time Systems Scheduling
