Model Extraction Attacks Revisited

TL;DR

This paper provides an in-depth analysis of how model extraction attack vulnerabilities in MLaaS platforms have evolved over seven years, challenging prior assumptions and offering insights for enhancing attack robustness.

Contribution

It offers a comprehensive characterization of current ME attack vulnerabilities, analyzes their evolution over four years, and suggests improvements for MLaaS security.

Findings

Many findings challenge previous reports on ME vulnerability.

ME vulnerability patterns are evolving over time.

Recommendations for improving MLaaS attack robustness.

Abstract

Model extraction (ME) attacks represent one major threat to Machine-Learning-as-a-Service (MLaaS) platforms by ``stealing'' the functionality of confidential machine-learning models through querying black-box APIs. Over seven years have passed since ME attacks were first conceptualized in the seminal work. During this period, substantial advances have been made in both ME attacks and MLaaS platforms, raising the intriguing question: How has the vulnerability of MLaaS platforms to ME attacks been evolving? In this work, we conduct an in-depth study to answer this critical question. Specifically, we characterize the vulnerability of current, mainstream MLaaS platforms to ME attacks from multiple perspectives including attack strategies, learning techniques, surrogate-model design, and benchmark tasks. Many of our findings challenge previously reported results, suggesting emerging patterns of ME vulnerability. Further, by analyzing the vulnerability of the same MLaaS platforms using historical datasets from the past four years, we retrospectively characterize the evolution of ME vulnerability over time, leading to a set of interesting findings. Finally, we make suggestions about improving the current practice of MLaaS in terms of attack robustness. Our study sheds light on the current state of ME vulnerability in the wild and points to several promising directions for future research.