Membership Inference Test: Auditing Training Data in Object Classification Models

TL;DR

This paper introduces a specialized Membership Inference Test (MINT) architecture for object recognition models, achieving 70-80% precision in identifying training data usage across large datasets, thereby enhancing model transparency.

Contribution

The paper presents a novel MINT architecture tailored for object recognition, improving inference accuracy and efficiency in auditing training data usage.

Findings

Achieved 70-80% precision in membership inference.

Demonstrated effectiveness across three public datasets with over 174K images.

Analyzed factors affecting MINT performance and transparency.

Abstract

In this research, we analyze the performance of Membership Inference Tests (MINT), focusing on determining whether given data were utilized during the training phase, specifically in the domain of object recognition. Within the area of object recognition, we propose and develop architectures tailored for MINT models. These architectures aim to optimize performance and efficiency in data utilization, offering a tailored solution to tackle the complexities inherent in the object recognition domain. We conducted experiments involving an object detection model, an embedding extractor, and a MINT module. These experiments were performed in three public databases, totaling over 174K images. The proposed architecture leverages convolutional layers to capture and model the activation patterns present in the data during the training process. Through our analysis, we are able to identify given data used for testing and training, achieving precision rates ranging between 70% and 80%, contingent upon the depth of the detection module layer chosen for input to the MINT module. Additionally, our studies entail an analysis of the factors influencing the MINT Module, delving into the contributing elements behind more transparent training processes.