Sionnx: Automatic Unit Test Generator for ONNX Conformance
Xinli Cai, Peng Zhou, Shuhan Ding, Guoyang Chen, Weifeng Zhang

TL;DR
Sionnx is an automatic unit test generator designed to verify ONNX operator compliance, using a new specification language and LLVM TableGen to achieve high coverage and cross-framework support.
Contribution
It introduces a novel Operator Specification Language and leverages LLVM TableGen to automatically generate comprehensive conformance tests for ONNX operators.
Findings
Enables automated verification of ONNX operator compliance.
Provides high coverage testing across multiple frameworks.
Open-sourced implementation available for community use.
Abstract
Open Neural Network Exchange (ONNX) is an open format to represent AI models and is supported by many machine learning frameworks. While ONNX defines unified and portable computation operators across various frameworks, the conformance tests for those operators are insufficient, which makes it difficult to verify if an operator's behavior in an ONNX backend implementation complies with the ONNX standard. In this paper, we present the first automatic unit test generator named Sionnx for verifying the compliance of ONNX implementation. First, we propose a compact yet complete set of rules to describe the operator's attributes and the properties of its operands. Second, we design an Operator Specification Language (OSL) to provide a high-level description for the operator's syntax. Finally, through this easy-to-use specification language, we are able to build a full testing specification…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdversarial Robustness in Machine Learning · Explainable Artificial Intelligence (XAI) · Topic Modeling
