Swift: High-Performance Sparse Tensor Contraction for Scientific Applications

TL;DR

Swift introduces a highly efficient algorithm for sparse tensor contraction that significantly reduces computation time by optimizing data structures and grouping methods, outperforming existing algorithms in scientific computing tasks.

Contribution

The paper presents Swift, a novel sparse tensor contraction algorithm that improves performance through better data structures and grouping, surpassing current methods.

Findings

Up to 20x speedup over state-of-the-art algorithms

Handles imbalanced tensors more effectively

Reduces unnecessary computations in tensor contraction

Abstract

In scientific fields such as quantum computing, physics, chemistry, and machine learning, high dimensional data are typically represented using sparse tensors. Tensor contraction is a popular operation on tensors to exploit meaning or alter the input tensors. Tensor contraction is, however, computationally expensive and grows quadratically with the number of elements. For this reason, specialized algorithms have been created to only operate on the nonzero elements. Current sparse tensor contraction algorithms utilize sub-optimal data structures that perform unnecessary computations which increase execution time and the overall time complexity. We propose Swift, a novel algorithm for sparse tensor contraction that replaces the costly sorting with more efficient grouping, utilizes better data structures to represent tensors, and employs more memory-friendly hash table implementation. Swift is evaluated against the state-of-the-art sparse tensor contraction algorithm, demonstrating up to 20x speedup in various test cases and being able to handle imbalanced input tensors significantly better.