High-Performance Level-1 and Level-2 BLAS

TL;DR

This paper presents new algorithms for level-1 and level-2 BLAS operations that significantly enhance performance by leveraging modern hardware features and compiler optimizations, without creating new libraries.

Contribution

It introduces novel algorithms for vector-vector and matrix-vector operations that outperform existing methods, emphasizing machine-oblivious design and compiler-based optimization.

Findings

Significant performance improvements in vector and matrix operations.

Algorithms rely on FMA, OpenMP, and compiler optimizations.

No new libraries introduced, only algorithmic enhancements.

Abstract

The introduction of the Basic Linear Algebra Subroutine (BLAS) in the 1970s paved the way for different libraries to solve the same problem with an improved approach and hardware. The new BLAS implementation led to High-Performance Computing (HPC) innovation. All the love went to the level 3 BLAS due to its humongous application in different fields, not bounded by computer science. However, level 1 and level 2 got neglected; we tried to solve the problem by introducing the new algorithm for the Vector-Vector dot product, Vector-Vector outer product and Matrix-Vector product, which improves the performance of these operations in a significant way. We are not introducing any library but algorithms, which improves upon the current state of art algorithms. Also, we rely on the FMA instruction, OpenMP, and the compiler to optimize the code rather than implementing the algorithm in assembly. Therefore, our current implementation is machine oblivious and depends on the compilers ability to optimize the code.