ShadowGenes: Leveraging Recurring Patterns within Computational Graphs for Model Genealogy

TL;DR

ShadowGenes is a signature-based method that analyzes computational graphs of neural networks to accurately identify their architecture and family, aiding model understanding and security assessment.

Contribution

It introduces a novel, format-agnostic signature-based technique for neural network model genealogy, validated on over 1,400 models with high accuracy.

Findings

Achieved 97.49% true positive rate

Attained 99.51% precision score

Demonstrated effectiveness on diverse models

Abstract

Machine learning model genealogy enables practitioners to determine which architectural family a neural network belongs to. In this paper, we introduce ShadowGenes, a novel, signature-based method for identifying a given model's architecture, type, and family. Our method involves building a computational graph of the model that is agnostic of its serialization format, then analyzing its internal operations to identify unique patterns, and finally building and refining signatures based on these. We highlight important workings of the underlying engine and demonstrate the technique used to construct a signature and scan a given model. This approach to model genealogy can be applied to model files without the need for additional external information. We test ShadowGenes on a labeled dataset of over 1,400 models and achieve a mean true positive rate of 97.49% and a precision score of 99.51%; which validates the technique as a practical method for model genealogy. This enables practitioners to understand the use cases of a given model, the internal computational process, and identify possible security risks, such as the potential for model backdooring.