Adaptive Domain Inference Attack with Concept Hierarchy

TL;DR

This paper introduces an adaptive domain inference attack (ADI) that leverages concept hierarchies to extract relevant training data from neural network models, even with minimal knowledge, improving attack effectiveness and efficiency.

Contribution

The paper proposes a novel adaptive attack method using concept hierarchies to infer training data, demonstrating its effectiveness and efficiency over existing methods.

Findings

ADI can successfully estimate relevant training data with minimal model access.

The extracted data significantly enhances model-inversion attack performance.

ADI converges faster and requires fewer model queries than other methods.

Abstract

With increasingly deployed deep neural networks in sensitive application domains, such as healthcare and security, it's essential to understand what kind of sensitive information can be inferred from these models. Most known model-targeted attacks assume attackers have learned the application domain or training data distribution to ensure successful attacks. Can removing the domain information from model APIs protect models from these attacks? This paper studies this critical problem. Unfortunately, even with minimal knowledge, i.e., accessing the model as an unnamed function without leaking the meaning of input and output, the proposed adaptive domain inference attack (ADI) can still successfully estimate relevant subsets of training data. We show that the extracted relevant data can significantly improve, for instance, the performance of model-inversion attacks. Specifically, the ADI method utilizes a concept hierarchy extracted from a collection of available public and private datasets and a novel algorithm to adaptively tune the likelihood of leaf concepts showing up in the unseen training data. We also designed a straightforward hypothesis-testing-based attack -- LDI. The ADI attack not only extracts partial training data at the concept level but also converges fastest and requires the fewest target-model accesses among all candidate methods. Our code is available at https://anonymous.4open.science/r/KDD-362D.