MiniTensor: A Lightweight, High-Performance Tensor Operations Library

TL;DR

MiniTensor is a minimalistic, high-performance tensor library with a PyTorch-like API, implemented in Rust, offering a small install footprint and essential features for CPU-based research and development.

Contribution

It introduces a lightweight tensor library with a Rust engine and Python API, focusing on minimalism, correctness, and performance, with significantly smaller size than mainstream frameworks.

Findings

MiniTensor has a package size of only a few megabytes.

It supports dense tensors, broadcasting, reductions, and neural network layers.

Performance is comparable to larger frameworks on CPU tasks.

Abstract

We present MiniTensor, an open source tensor operations library that focuses on minimalism, correctness, and performance. MiniTensor exposes a familiar PyTorch-like Python API while it executes performance critical code in a Rust engine. The core supports dense $n$ dimensional tensors, broadcasting, reductions, matrix multiplication, reverse mode automatic differentiation, a compact set of neural network layers, and standard optimizers. In this paper, we describe the design of MiniTensor's architecture, including its efficient memory management, dynamic computation graph for gradients, and integration with Python via PyO3. We also compare the install footprint with PyTorch and TensorFlow to demonstrate that MiniTensor achieves a package size of only a few megabytes, several orders of magnitude smaller than mainstream frameworks, while preserving the essentials needed for research and development on CPUs. The repository can be found at https://github.com/neuralsorcerer/minitensor