Pipeflow: An Efficient Task-Parallel Pipeline Programming Framework using Modern C++

TL;DR

Pipeflow is a new C++ framework that improves task-parallel pipeline execution by focusing on scheduling rather than data abstractions, leading to better performance in pipeline algorithms.

Contribution

It introduces a task-parallel pipeline framework that emphasizes scheduling efficiency over data abstractions, outperforming existing frameworks like oneTBB.

Findings

Pipeflow outperforms oneTBB by 24% in VLSI placement.

Pipeflow is 10% faster in timing analysis workloads.

Demonstrates improved efficiency in real-world pipeline applications.

Abstract

Pipeline is a fundamental parallel programming pattern. Mainstream pipeline programming frameworks count on data abstractions to perform pipeline scheduling. This design is convenient for data-centric pipeline applications but inefficient for algorithms that only exploit task parallelism in pipeline. As a result, we introduce a new task-parallel pipeline programming framework called Pipeflow. Pipeflow does not design yet another data abstraction but focuses on the pipeline scheduling itself, enabling more efficient implementation of task-parallel pipeline algorithms than existing frameworks. We have evaluated Pipeflow on both micro-benchmarks and real-world applications. As an example, Pipeflow outperforms oneTBB 24% and 10% faster in a VLSI placement and a timing analysis workloads that adopt pipeline parallelism to speed up runtimes, respectively.