Tensor Data Scattering and the Impossibility of Slicing Theorem

TL;DR

This paper introduces a theoretical framework for sparse tensor data scattering, presents a key theorem on slicing impossibility, and offers a sparsity measure to evaluate storage efficiency and parallelization potential.

Contribution

It establishes a standard representation for sparse tensors, proves a theorem on slicing limitations, and provides a sparsity metric for performance analysis.

Findings

Theorem reveals fundamental slicing limitations in tensor data scattering.

A sparsity measure effectively indicates storage efficiency and parallelization potential.

Open-source CUDA code supports practical implementation and testing.

Abstract

This paper proposes a standard way to represent sparse tensors. A broad theoretical framework for tensor data scattering methods used in various deep learning frameworks is established. This paper presents a theorem that is very important for performance analysis and accelerator optimization for implementing data scattering. The theorem shows how the impossibility of slicing happens in tenser data scattering. A sparsity measuring formula is provided, which can effectively indicate the storage efficiency of sparse tensor and the possibility of parallelly using it. The source code, including CUDA code, is provided in a related open-source project.