GEMMFIP: Unifying GEMM in BLIS

TL;DR

This paper introduces GEMMFIP, a unified approach for high-performance matrix multiplication across small and large matrices by fusing packing with computation, simplifying tuning and improving efficiency.

Contribution

It proposes a novel technique that unifies optimization strategies for small and large matrices in GEMM, implemented within the BLIS framework.

Findings

Achieves high performance for both small and large matrices

Simplifies tuning of general-purpose matrix libraries

Demonstrates effectiveness across multiple architectures

Abstract

Matrix libraries often focus on achieving high performance for problems considered to be either "small" or "large", as these two scenarios tend to respond best to different optimization strategies. We propose a unified technique for implementing matrix operations like general matrix multiplication (GEMM) that can achieve high performance for both small and large problem sizes. The key is to fuse packing -- an operation that copies data to a contiguous layout in memory and which is critical for large matrix performance -- with the first computational "pass" over that data. This boosts performance across the problem size spectrum. As a result, tuning general-purpose libraries becomes simpler since it obviates the need to carefully express and parameterize logic that chooses between a "small matrix" strategy and a "large matrix" strategy. A prototype implementation of the technique built with the BLAS-like Library Instantiation Software (BLIS) framework is described and performance on a range of architectures is reported.