A Parallel Sparse Tensor Benchmark Suite on CPUs and GPUs
Jiajia Li, Mahesh Lakshminarasimhan, Xiaolong Wu, Ang Li and, Catherine Olschanowsky, Kevin Barker

TL;DR
This paper introduces a comprehensive benchmark suite for sparse tensor computations on CPUs and GPUs, facilitating performance analysis and optimization across various applications.
Contribution
It presents a new benchmark suite with reference implementations for arbitrary-order sparse tensor kernels using COO and HiCOO formats, applicable to real-world and synthetic tensors.
Findings
Provides Roofline performance models for sparse tensor kernels
Enables performance comparison across CPU and GPU platforms
Supports real-world and synthetic tensor data
Abstract
Tensor computations present significant performance challenges that impact a wide spectrum of applications ranging from machine learning, healthcare analytics, social network analysis, data mining to quantum chemistry and signal processing. Efforts to improve the performance of tensor computations include exploring data layout, execution scheduling, and parallelism in common tensor kernels. This work presents a benchmark suite for arbitrary-order sparse tensor kernels using state-of-the-art tensor formats: coordinate (COO) and hierarchical coordinate (HiCOO) on CPUs and GPUs. It presents a set of reference tensor kernel implementations that are compatible with real-world tensors and power law tensors extended from synthetic graph generation techniques. We also propose Roofline performance models for these kernels to provide insights of computer platforms from sparse tensor view.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTensor decomposition and applications · Parallel Computing and Optimization Techniques · Advanced Neuroimaging Techniques and Applications
