Generating Highly Structured Test Inputs Leveraging Constraint-Guided Graph Refinement

TL;DR

This paper introduces GRAphRef, a graph-based framework for generating valid, structured test inputs for AI systems processing complex data like 3D meshes, improving validity and semantic consistency through constraint-guided mutation and refinement.

Contribution

It proposes a unified, graph-based approach for test input generation that enforces structural constraints, enhancing validity and applicability across diverse structured data domains.

Findings

GRAphRef improves structural validity over baseline methods.

Semantic preservation is enhanced with the proposed approach.

The framework demonstrates acceptable performance overhead.

Abstract

[Context] Modern AI applications increasingly process highly structured data, such as 3D meshes and point clouds, where test input generation must preserve both structural and semantic validity. However, existing fuzzing tools and input generators are typically handcrafted for specific input types and often generate invalid inputs that are subsequently discarded, leading to inefficiency and poor generalizability. [Objective] This study investigates whether test inputs for structured domains can be unified through a graph-based representation, enabling general, reusable mutation strategies while enforcing structural constraints. We will evaluate the effectiveness of this approach in enhancing input validity and semantic preservation across eight AI systems. [Method] We develop and evaluate GRAphRef, a graph-based test input generation framework that supports constraint-based mutation and refinement. GRAphRef maps structured inputs to graphs, applies neighbor-similarity-guided mutations, and uses a constraint-refinement phase to repair invalid inputs. We will conduct a confirmatory study across eight real-world mesh-processing AI systems, comparing GRAphRef with AFL, MeshAttack, Saffron, and two ablated variants. Evaluation metrics include structural validity, semantic preservation (via prediction consistency), and performance overhead. Experimental data is derived from ShapeNetCore mesh seeds and model outputs from systems like MeshCNN and HodgeNet. Statistical analysis and component latency breakdowns will be used to assess each hypothesis.