Piracy Resistant Watermarks for Deep Neural Networks

TL;DR

This paper introduces null embedding, a novel method for embedding watermarks into deep neural networks at initial training, making them resistant to piracy and tampering, and effective across various models and tasks.

Contribution

The paper presents null embedding, a new approach that embeds watermarks during initial training to prevent piracy and removal, addressing limitations of incremental training methods.

Findings

Watermarks resist removal through tuning or incremental training.

Watermarks remain robust after model fine-tuning, compression, and backdoor detection.

Watermarked models support transfer learning without losing watermark integrity.

Abstract

As companies continue to invest heavily in larger, more accurate and more robust deep learning models, they are exploring approaches to monetize their models while protecting their intellectual property. Model licensing is promising, but requires a robust tool for owners to claim ownership of models, i.e. a watermark. Unfortunately, current designs have not been able to address piracy attacks, where third parties falsely claim model ownership by embedding their own "pirate watermarks" into an already-watermarked model. We observe that resistance to piracy attacks is fundamentally at odds with the current use of incremental training to embed watermarks into models. In this work, we propose null embedding, a new way to build piracy-resistant watermarks into DNNs that can only take place at a model's initial training. A null embedding takes a bit string (watermark value) as input, and builds strong dependencies between the model's normal classification accuracy and the watermark. As a result, attackers cannot remove an embedded watermark via tuning or incremental training, and cannot add new pirate watermarks to already watermarked models. We empirically show that our proposed watermarks achieve piracy resistance and other watermark properties, over a wide range of tasks and models. Finally, we explore a number of adaptive counter-measures, and show our watermark remains robust against a variety of model modifications, including model fine-tuning, compression, and existing methods to detect/remove backdoors. Our watermarked models are also amenable to transfer learning without losing their watermark properties.