Copernicus: Characterizing the Performance Implications of Compression Formats Used in Sparse Workloads

TL;DR

This paper evaluates how different sparse matrix formats impact performance on domain-specific architectures, specifically using FPGA-based implementations of sparse matrix-vector multiplication, highlighting the trade-offs in decompression and processing efficiency.

Contribution

It provides a comprehensive analysis of seven common sparse formats' performance implications on FPGA-based DSAs, optimizing decompression for fair comparison and exploring multiple metrics.

Findings

Decompression overhead varies significantly across formats.

Certain formats achieve higher throughput but at increased resource cost.

Performance trade-offs depend on workload characteristics and format choice.

Abstract

Sparse matrices are the key ingredients of several application domains, from scientific computation to machine learning. The primary challenge with sparse matrices has been efficiently storing and transferring data, for which many sparse formats have been proposed to significantly eliminate zero entries. Such formats, essentially designed to optimize memory footprint, may not be as successful in performing faster processing. In other words, although they allow faster data transfer and improve memory bandwidth utilization -- the classic challenge of sparse problems -- their decompression mechanism can potentially create a computation bottleneck. Not only is this challenge not resolved, but also it becomes more serious with the advent of domain-specific architectures (DSAs), as they intend to more aggressively improve performance. The performance implications of using various formats along with DSAs, however, has not been extensively studied by prior work. To fill this gap of knowledge, we characterize the impact of using seven frequently used sparse formats on performance, based on a DSA for sparse matrix-vector multiplication (SpMV), implemented on an FPGA using high-level synthesis (HLS) tools, a growing and popular method for developing DSAs. Seeking a fair comparison, we tailor and optimize the HLS implementation of decompression for each format. We thoroughly explore diverse metrics, including decompression overhead, latency, balance ratio, throughput, memory bandwidth utilization, resource utilization, and power consumption, on a variety of real-world and synthetic sparse workloads.