NaN-Propagation: A Novel Method for Sparsity Detection in Black-Box Computational Functions

TL;DR

NaN-Propagation introduces a new sparsity detection method for black-box functions that uses NaN contamination to accurately identify zero-gradient dependencies, significantly improving computational efficiency.

Contribution

The paper presents NaN-Propagation, a novel approach leveraging IEEE 754 NaN propagation to detect sparsity patterns in black-box functions without code modifications.

Findings

Achieved 1.52x speedup in an aerospace model

Uncovered dependencies missed by traditional methods

Enabled faster-than-linear sparsity detection algorithms

Abstract

When numerically evaluating a function's gradient, sparsity detection can enable substantial computational speedups through Jacobian coloring and compression. However, sparsity detection techniques for black-box functions are limited, and existing finite-difference-based methods suffer from false negatives due to coincidental zero gradients. These false negatives can silently corrupt gradient calculations, leading to difficult-to-diagnose errors. We introduce NaN-propagation, which exploits the universal contamination property of IEEE 754 Not-a-Number values to trace input-output dependencies through floating-point numerical computations. By systematically contaminating inputs with NaN and observing which outputs become NaN, the method reconstructs conservative sparsity patterns that eliminate a major source of false negatives. We demonstrate this approach on an aerospace wing weight model, achieving a 1.52x speedup while uncovering dozens of dependencies missed by conventional methods -- a significant practical improvement since gradient computation is often the bottleneck in optimization workflows. The technique leverages IEEE 754 compliance to work across programming languages and math libraries without requiring modifications to existing black-box codes. Furthermore, advanced strategies such as NaN payload encoding via direct bit manipulation enable faster-than-linear time complexity, yielding speed improvements over existing black-box sparsity detection methods. Practical algorithms are also proposed to mitigate challenges from branching code execution common in engineering applications.