FuzzyFlow: Leveraging Dataflow To Find and Squash Program Optimization Bugs

TL;DR

FuzzyFlow is a framework that uses dataflow analysis to efficiently identify and fix bugs in program optimizations, significantly reducing testing time and improving robustness.

Contribution

It introduces a novel dataflow-based approach for fault localization and test case extraction in program optimization verification.

Findings

Up to 528 times faster optimization testing.

Effective minimization of test inputs with trade-offs.

Demonstrated on real-world applications.

Abstract

The current hardware landscape and application scale is driving performance engineers towards writing bespoke optimizations. Verifying such optimizations, and generating minimal failing cases, is important for robustness in the face of changing program conditions, such as inputs and sizes. However, isolation of minimal test-cases from existing applications and generating new configurations are often difficult due to side effects on the system state, mostly related to dataflow. This paper introduces FuzzyFlow: a fault localization and test case extraction framework designed to test program optimizations. We leverage dataflow program representations to capture a fully reproducible system state and area-of-effect for optimizations to enable fast checking for semantic equivalence. To reduce testing time, we design an algorithm for minimizing test inputs, trading off memory for recomputation. We demonstrate FuzzyFlow on example use cases in real-world applications where the approach provides up to 528 times faster optimization testing and debugging compared to traditional approaches.