Transferable Watermarking to Self-supervised Pre-trained Graph Encoders by Trigger Embeddings

TL;DR

This paper introduces a watermarking method for self-supervised graph encoders that embeds a unique trigger pattern into the model, enabling copyright protection and verification across multiple downstream tasks.

Contribution

A novel watermarking framework for GSSL graph encoders that embeds trigger-based backdoors for copyright protection and verification.

Findings

Watermark transfers effectively to downstream tasks like node classification and link prediction.

The watermarking approach maintains high model fidelity and robustness.

The method enables reliable black-box watermark verification in practice.

Abstract

Recent years have witnessed the prosperous development of Graph Self-supervised Learning (GSSL), which enables to pre-train transferable foundation graph encoders. However, the easy-to-plug-in nature of such encoders makes them vulnerable to copyright infringement. To address this issue, we develop a novel watermarking framework to protect graph encoders in GSSL settings. The key idea is to force the encoder to map a set of specially crafted trigger instances into a unique compact cluster in the outputted embedding space during model pre-training. Consequently, when the encoder is stolen and concatenated with any downstream classifiers, the resulting model inherits the `backdoor' of the encoder and predicts the trigger instances to be in a single category with high probability regardless of the ground truth. Experimental results have shown that, the embedded watermark can be transferred to various downstream tasks in black-box settings, including node classification, link prediction and community detection, which forms a reliable watermark verification system for GSSL in reality. This approach also shows satisfactory performance in terms of model fidelity, reliability and robustness.