A new membership inference attack that spots memorization in generative and predictive models: Loss-Based with Reference Model algorithm (LBRM)

TL;DR

This paper introduces the LBRM algorithm, a novel membership inference attack that effectively detects memorization in generative and predictive models, significantly improving privacy risk assessment in time series imputation.

Contribution

The paper presents a new LBRM method utilizing a reference model to enhance membership inference accuracy, demonstrating substantial improvements over existing techniques.

Findings

AUROC improved by ~40% without fine-tuning

AUROC increased by ~60% with fine-tuning

Validated on multiple time series imputation architectures

Abstract

Generative models can unintentionally memorize training data, posing significant privacy risks. This paper addresses the memorization phenomenon in time series imputation models, introducing the Loss-Based with Reference Model (LBRM) algorithm. The LBRM method leverages a reference model to enhance the accuracy of membership inference attacks, distinguishing between training and test data. Our contributions are twofold: first, we propose an innovative method to effectively extract and identify memorized training data, significantly improving detection accuracy. On average, without fine-tuning, the AUROC improved by approximately 40\%. With fine-tuning, the AUROC increased by approximately 60\%. Second, we validate our approach through membership inference attacks on two types of architectures designed for time series imputation, demonstrating the robustness and versatility of the LBRM approach in different contexts. These results highlight the significant enhancement in detection accuracy provided by the LBRM approach, addressing privacy risks in time series imputation models.