Tensors Fitting Perfectly

TL;DR

The paper introduces Tensors Fitting Perfectly, a static analysis tool for reasoning about NDArray shapes in Swift for TensorFlow, helping detect inconsistencies and providing shape insights to improve program correctness.

Contribution

It presents a novel static analysis method that synthesizes shape constraints from abstract interpretation to verify and understand NDArray shapes in Swift programs.

Findings

Successfully detects shape inconsistencies in programs.

Provides insights into intermediate NDArray shapes.

Enhances program correctness with combined static and runtime analysis.

Abstract

Multidimensional arrays (NDArrays) are a central abstraction in modern scientific computing environments. Unfortunately, they can make reasoning about programs harder as the number of different array shapes used in an execution of a program is usually very large, and they rarely appear explicitly in program text. To make things worse, many operators make implicit assumptions about the shapes of their inputs: array addition is commonly enriched with broadcasting semantics, while matrix multiplication assumes that the lengths of contracted dimensions are equal. Because precise reasoning about shapes is crucial to write correct programs using NDArrays, and because shapes are often hard to infer from a quick glance at the program, we developed Tensors Fitting Perfectly, a static analysis tool that reasons about NDArray shapes in Swift for TensorFlow programs by synthesizing a set of shape constraints from an abstract interpretation of the program. It can both (1) check for possible inconsistencies, and (2) provide direct insights about the shapes of intermediate values appearing in the program, including via a mechanism called shape holes. The static analysis works in concert with optional runtime assertions to improve the productivity of program authors.