ALTO: Adaptive Linearized Storage of Sparse Tensors

TL;DR

ALTO introduces a versatile sparse tensor storage format that improves processing efficiency and memory usage, significantly outperforming existing formats in tensor decomposition tasks.

Contribution

The paper presents ALTO, a novel mode-agnostic sparse tensor format with adaptive encoding that enhances performance and reduces memory footprint.

Findings

ALTO achieves an 8X speedup over state-of-the-art mode-agnostic formats.

ALTO provides a 4.3X better compression ratio compared to mode-specific formats.

ALTO reduces workload imbalance and synchronization overhead in tensor computations.

Abstract

The analysis of high-dimensional sparse data is becoming increasingly popular in many important domains. However, real-world sparse tensors are challenging to process due to their irregular shapes and data distributions. We propose the Adaptive Linearized Tensor Order (ALTO) format, a novel mode-agnostic (general) representation that keeps neighboring nonzero elements in the multi-dimensional space close to each other in memory. To generate the indexing metadata, ALTO uses an adaptive bit encoding scheme that trades off index computations for lower memory usage and more effective use of memory bandwidth. Moreover, by decoupling its sparse representation from the irregular spatial distribution of nonzero elements, ALTO eliminates the workload imbalance and greatly reduces the synchronization overhead of tensor computations. As a result, the parallel performance of ALTO-based tensor operations becomes a function of their inherent data reuse. On a gamut of tensor datasets, ALTO outperforms an oracle that selects the best state-of-the-art format for each dataset, when used in key tensor decomposition operations. Specifically, ALTO achieves a geometric mean speedup of 8X over the best mode-agnostic (coordinate and hierarchical coordinate) formats, while delivering a geometric mean compression ratio of 4.3X relative to the best mode-specific (compressed sparse fiber) formats.