Beyond Labeling Oracles: What does it mean to steal ML models?

TL;DR

This paper critically examines model extraction attacks, revealing that prior knowledge of data distribution is more crucial than attack strategy, and challenges existing evaluation methods of attack success.

Contribution

It provides a comprehensive analysis of factors influencing model extraction success, emphasizing the importance of prior data knowledge over attack policies.

Findings

Prior knowledge of in-distribution data is the dominant factor in attack success.

Current evaluation methods misinterpret the effectiveness of model extraction attacks.

Assumptions about cost savings in data acquisition are often invalid in practice.

Abstract

Model extraction attacks are designed to steal trained models with only query access, as is often provided through APIs that ML-as-a-Service providers offer. Machine Learning (ML) models are expensive to train, in part because data is hard to obtain, and a primary incentive for model extraction is to acquire a model while incurring less cost than training from scratch. Literature on model extraction commonly claims or presumes that the attacker is able to save on both data acquisition and labeling costs. We thoroughly evaluate this assumption and find that the attacker often does not. This is because current attacks implicitly rely on the adversary being able to sample from the victim model's data distribution. We thoroughly research factors influencing the success of model extraction. We discover that prior knowledge of the attacker, i.e., access to in-distribution data, dominates other factors like the attack policy the adversary follows to choose which queries to make to the victim model API. Our findings urge the community to redefine the adversarial goals of ME attacks as current evaluation methods misinterpret the ME performance.