DREAM: Domain-agnostic Reverse Engineering Attributes of Black-box Model

TL;DR

This paper introduces DREAM, a domain-agnostic framework for reverse engineering attributes of black-box models without access to their training data, using out-of-distribution generalization to achieve broad applicability.

Contribution

The paper proposes a novel OOD-based meta-model approach for black-box model attribute inference that does not require training data access, enhancing generalization across domains.

Findings

DREAM outperforms baseline methods in attribute inference accuracy.

The framework demonstrates strong generalization to unseen domains.

Experimental results validate the effectiveness of the proposed approach.

Abstract

Deep learning models are usually black boxes when deployed on machine learning platforms. Prior works have shown that the attributes (e.g., the number of convolutional layers) of a target black-box model can be exposed through a sequence of queries. There is a crucial limitation: these works assume the training dataset of the target model is known beforehand and leverage this dataset for model attribute attack. However, it is difficult to access the training dataset of the target black-box model in reality. Therefore, whether the attributes of a target black-box model could be still revealed in this case is doubtful. In this paper, we investigate a new problem of black-box reverse engineering, without requiring the availability of the target model's training dataset. We put forward a general and principled framework DREAM, by casting this problem as out-of-distribution (OOD) generalization. In this way, we can learn a domain-agnostic meta-model to infer the attributes of the target black-box model with unknown training data. This makes our method one of the kinds that can gracefully apply to an arbitrary domain for model attribute reverse engineering with strong generalization ability. Extensive experimental results demonstrate the superiority of our proposed method over the baselines.