Exploring prompts to elicit memorization in masked language model-based named entity recognition

TL;DR

This study investigates how different prompts affect the detection of memorization in masked language model-based NER models, revealing significant variability and factors influencing prompt effectiveness.

Contribution

It introduces a systematic analysis of prompt effects on memorization detection in NER models, highlighting prompt properties and model dependencies.

Findings

Prompt performance varies by up to 16 percentage points across prompts.

Prompt engineering can significantly enhance memorization detection.

Model-dependent performance that generalizes across name sets.

Abstract

Training data memorization in language models impacts model capability (generalization) and safety (privacy risk). This paper focuses on analyzing prompts' impact on detecting the memorization of 6 masked language model-based named entity recognition models. Specifically, we employ a diverse set of 400 automatically generated prompts, and a pairwise dataset where each pair consists of one person's name from the training set and another name out of the set. A prompt completed with a person's name serves as input for getting the model's confidence in predicting this name. Finally, the prompt performance of detecting model memorization is quantified by the percentage of name pairs for which the model has higher confidence for the name from the training set. We show that the performance of different prompts varies by as much as 16 percentage points on the same model, and prompt engineering further increases the gap. Moreover, our experiments demonstrate that prompt performance is model-dependent but does generalize across different name sets. A comprehensive analysis indicates how prompt performance is influenced by prompt properties, contained tokens, and the model's self-attention weights on the prompt.