Effectiveness of Distillation Attack and Countermeasure on Neural Network Watermarking

TL;DR

This paper investigates the vulnerability of neural network watermarking to distillation attacks and proposes a new regularization method, ingrain, to enhance watermark robustness against such attacks.

Contribution

The paper introduces ingrain, a novel regularization technique that improves watermark robustness in neural networks against distillation attacks.

Findings

Ingrain significantly increases watermark robustness against distillation.

Distillation effectively removes existing watermarks due to their redundancy.

Ingrain maintains robustness against other transformation techniques.

Abstract

The rise of machine learning as a service and model sharing platforms has raised the need of traitor-tracing the models and proof of authorship. Watermarking technique is the main component of existing methods for protecting copyright of models. In this paper, we show that distillation, a widely used transformation technique, is a quite effective attack to remove watermark embedded by existing algorithms. The fragility is due to the fact that distillation does not retain the watermark embedded in the model that is redundant and independent to the main learning task. We design ingrain in response to the destructive distillation. It regularizes a neural network with an ingrainer model, which contains the watermark, and forces the model to also represent the knowledge of the ingrainer. Our extensive evaluations show that ingrain is more robust to distillation attack and its robustness against other widely used transformation techniques is comparable to existing methods.