FBI: Fingerprinting models with Benign Inputs

TL;DR

This paper introduces robust fingerprinting methods for deep neural networks that use unmodified, benign inputs to identify and verify models, even after significant modifications, validated on over 1,000 networks.

Contribution

It proposes new fingerprinting schemes resilient to model modifications and extends the task to identifying model families using benign inputs, leveraging information theory and greedy algorithms.

Findings

Effective fingerprinting on over 1,000 networks

Resilience to model modifications like retraining and quantization

Successful identification of model families using benign inputs

Abstract

Recent advances in the fingerprinting of deep neural networks detect instances of models, placed in a black-box interaction scheme. Inputs used by the fingerprinting protocols are specifically crafted for each precise model to be checked for. While efficient in such a scenario, this nevertheless results in a lack of guarantee after a mere modification (like retraining, quantization) of a model. This paper tackles the challenges to propose i) fingerprinting schemes that are resilient to significant modifications of the models, by generalizing to the notion of model families and their variants, ii) an extension of the fingerprinting task encompassing scenarios where one wants to fingerprint not only a precise model (previously referred to as a detection task) but also to identify which model family is in the black-box (identification task). We achieve both goals by demonstrating that benign inputs, that are unmodified images, for instance, are sufficient material for both tasks. We leverage an information-theoretic scheme for the identification task. We devise a greedy discrimination algorithm for the detection task. Both approaches are experimentally validated over an unprecedented set of more than 1,000 networks.