Tensors and n-d Arrays:A Mathematics of Arrays (MoA), psi-Calculus and the Composition of Tensor and Array Operations

TL;DR

This paper explores the mathematical foundations and optimal implementation strategies for tensor and array operations, especially the Kronecker product, using MoA and psi-Calculus to enhance performance and reproducibility.

Contribution

It introduces a novel framework combining MoA and psi-Calculus for efficient, scalable, and verifiable tensor and array computations, addressing data structure and indexing challenges.

Findings

Optimal implementation strategies for Kronecker products across hardware architectures

Use of MoA and psi-Calculus improves scalability and reproducibility

Framework supports complex tensor and array operation composition

Abstract

The Kronecker product is a key algorithm and is ubiquitous across the physical, biological, and computation social sciences. Thus considerations of optimal implementation are important. The need to have high performance and computational reproducibility is paramount. Moreover, due to the need to compose multiple Kronecker products, issues related to data structures, layout and indexing algebra require a new look at an old problem. This paper discusses the outer product/tensor product and a special case of the tensor product: the Kronecker product, along with optimal implementation when composed, and mapped to complex processor/memory hierarchies. We discuss how the use of ``A Mathematics of Arrays" (MoA), and the psi-Calculus, (a calculus of indexing with shapes), provides optimal, verifiable, reproducible, scalable, and portable implementations of both hardware and software.