Optimum Depth of the Bounded Pipeline

TL;DR

This paper develops analytical models to determine the optimal depth of bounded computational pipelines, accounting for structural hazards, and validates these models through computer simulations.

Contribution

It introduces new formulas for optimal pipeline depth considering hazards and proposes simplified models for practical calculation.

Findings

The optimal pipeline depth can vary significantly with small data changes.

A pipeline with a single hazard provides a more stable approximation of processing time.

Models are validated through multi-threaded simulation and algebraic methods.

Abstract

The paper is devoted to studying the performance of a computational pipeline, the number of simultaneously executing stages of which at each time is bounded from above by a fixed number. A look at the restriction as a structural hazard makes it possible to construct an analytical model for calculating the processing time of a given input data amount. Using this model, led to a formula for calculating the optimal depth of a bounded pipeline for a given volume of input data. The formula shows that the optimal depth can get large changes for small changes in the amount of data. To eliminate this disadvantage and to obtain a more convenient formula for optimal depth, a pipeline with a single random hazard is constructed, the mathematical expectation of a random value of the processing time of which approximates the analytical model of the bounded pipeline. In addition, a pipeline with two hazards has been built, the analytical model of which allowed obtaining formulas for calculating the optimal depth of a bounded pipeline with restart for a given amount of data. To check whether the proposed analytical models are consistent with the experiments to calculate the processing time, two methods of computer simulation of bounded pipelines are used, the first of which is constructed as a multi-threaded application, and the second is based on the theory of free partially commutative monoids.