The Continuous Tensor Abstraction: Where Indices are Real

TL;DR

This paper introduces the continuous tensor abstraction, enabling tensor indices over real numbers, supporting infinite domains, and demonstrating significant performance improvements in applications like geometry, graphics, and neural rendering.

Contribution

It presents the first continuous tensor model, a new tensor format, and a code generation technique, expanding tensor programming to continuous domains with practical efficiency.

Findings

Achieves 9.20x speedup on 2D radius search

Reduces code lines by up to 60x in applications

Demonstrates competitive performance in diverse applications

Abstract

This paper introduces the continuous tensor abstraction, allowing indices to take real-number values (for example, A[3.14]). It also presents continuous tensor algebra expressions, such as C(x,y) = A(x,y) * B(x,y), where indices are defined over a continuous domain. This work expands the traditional tensor model to include continuous tensors. Our implementation supports piecewise-constant tensors, enabling infinite domains to be processed in finite time. We also introduce a new tensor format for efficient storage and a code generation technique for automatic kernel generation. For the first time, our abstraction expresses domains like computational geometry and computer graphics in the language of tensor programming. Our approach demonstrates competitive or better performance than hand-optimized kernels in leading libraries across diverse applications. Compared to hand-implemented libraries on a CPU, our compiler-based implementation achieves an average speedup of 9.20x on 2D radius search with approximately 60x fewer lines of code (LoC), 1.22x on genomic interval overlapping queries (with approximately 18x LoC saving), and 1.69x on trilinear interpolation in Neural Radiance Field (with approximately 6x LoC saving).