Discriminating Equivalent Algorithms via Relative Performance

TL;DR

This paper introduces a measurement-based clustering method to reliably identify the subset of algorithms that are consistently faster than others, despite performance fluctuations, enabling robust algorithm selection in scientific computing.

Contribution

It proposes a novel relative performance clustering approach that improves robustness in identifying top algorithms under noisy conditions, facilitating automatic selection.

Findings

Robust identification of faster algorithms using pair-wise performance comparisons.

Effective clustering of algorithms into performance classes despite noise.

Development of machine learning models for automatic algorithm selection.

Abstract

In scientific computing, it is common that a mathematical expression can be computed by many different algorithms (sometimes over hundreds), each identifying a specific sequence of library calls. Although mathematically equivalent, those algorithms might exhibit significant differences in terms of performance. However in practice, due to fluctuations, there is not one algorithm that consistently performs noticeably better than the rest. For this reason, with this work we aim to identify not the one best algorithm, but the subset of algorithms that are reliably faster than the rest. To this end, instead of using the usual approach of quantifying the performance of an algorithm in absolute terms, we present a measurement-based clustering approach to sort the algorithms into equivalence (or performance) classes using pair-wise comparisons. We show that this approach, based on relative performance, leads to robust identification of the fastest algorithms even under noisy system conditions. Furthermore, it enables the development of practical machine learning models for automatic algorithm selection.