Dataset Inference for Self-Supervised Models

TL;DR

This paper proposes a dataset inference method to detect stolen self-supervised models by analyzing output likelihoods, offering a new defense mechanism against model stealing attacks in vision applications.

Contribution

It introduces a novel dataset inference defense leveraging log-likelihood differences to identify stolen models, addressing a gap in existing defenses for self-supervised encoders.

Findings

Effective detection of stolen encoders using log-likelihood analysis

High fidelity in identifying stolen models without downstream task involvement

Promising results in vision domain demonstrating defense viability

Abstract

Self-supervised models are increasingly prevalent in machine learning (ML) since they reduce the need for expensively labeled data. Because of their versatility in downstream applications, they are increasingly used as a service exposed via public APIs. At the same time, these encoder models are particularly vulnerable to model stealing attacks due to the high dimensionality of vector representations they output. Yet, encoders remain undefended: existing mitigation strategies for stealing attacks focus on supervised learning. We introduce a new dataset inference defense, which uses the private training set of the victim encoder model to attribute its ownership in the event of stealing. The intuition is that the log-likelihood of an encoder's output representations is higher on the victim's training data than on test data if it is stolen from the victim, but not if it is independently trained. We compute this log-likelihood using density estimation models. As part of our evaluation, we also propose measuring the fidelity of stolen encoders and quantifying the effectiveness of the theft detection without involving downstream tasks; instead, we leverage mutual information and distance measurements. Our extensive empirical results in the vision domain demonstrate that dataset inference is a promising direction for defending self-supervised models against model stealing.