Tensor Algebra on an Optoelectronic Microchip

TL;DR

This paper introduces a novel approach to accelerate tensor algebra computations by utilizing an optoelectronic microchip, including a specialized programming language and memory-efficient storage methods optimized for optical hardware.

Contribution

It presents a new optical hardware-based method for tensor algebra acceleration, along with a dedicated programming language and optimized sparse tensor storage techniques.

Findings

Outperforms conventional array storage in memory efficiency

Enables native execution of complex tensor operations on optical hardware

Demonstrates potential for significant speedups in tensor computations

Abstract

Tensor algebra lies at the core of computational science and machine learning. Due to its high usage, entire libraries exist dedicated to improving its performance. Conventional tensor algebra performance boosts focus on algorithmic optimizations, which in turn lead to incremental improvements. In this paper, we describe a method to accelerate tensor algebra a different way: by outsourcing operations to an optical microchip. We outline a numerical programming language developed to perform tensor algebra computations that is designed to leverage our optical hardware's full potential. We introduce the language's current grammar and go over the compiler design. We then show a new way to store sparse rank-n tensors in RAM that outperforms conventional array storage (used by C++, Java, etc.). This method is more memory-efficient than Compressed Sparse Fiber (CSF) format and is specifically tuned for our optical hardware. Finally, we show how the scalar-tensor product, rank-$n$ Kronecker product, tensor dot product, Khatri-Rao product, face-splitting product, and vector cross product can be compiled into operations native to our optical microchip through various tensor decompositions.