Compiling Structured Tensor Algebra
Mahdi Ghorbani, Mathieu Huot, Shideh Hashemian, Amir Shaikhha

TL;DR
This paper introduces StructTensor, a framework that symbolically computes tensor structures at compile time, enabling optimized dense and sparse tensor algebra computations with improved performance.
Contribution
It presents STUR, an intermediate language for capturing tensor computations and their structures, and demonstrates how symbolic structure computation enhances efficiency.
Findings
Outperforms state-of-the-art frameworks in tensor algebra tasks.
Effectively captures sparsity and redundancy for optimization.
Provides a sound mathematical basis for lossless tensor computation.
Abstract
Tensor algebra is essential for data-intensive workloads in various computational domains. Computational scientists face a trade-off between the specialization degree provided by dense tensor algebra and the algorithmic efficiency that leverages the structure provided by sparse tensors. This paper presents StructTensor, a framework that symbolically computes structure at compilation time. This is enabled by Structured Tensor Unified Representation (STUR), an intermediate language that can capture tensor computations as well as their sparsity and redundancy structures. Through a mathematical view of lossless tensor computations, we show that our symbolic structure computation and the related optimizations are sound. Finally, for different tensor computation workloads and structures, we experimentally show how capturing the symbolic structure can result in outperforming state-of-the-art…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsParallel Computing and Optimization Techniques · Computational Physics and Python Applications · Tensor decomposition and applications
