Quantifying Association Capabilities of Large Language Models and Its Implications on Privacy Leakage

TL;DR

This paper investigates how large language models' increasing scale enhances their ability to associate information, highlighting privacy risks related to personally identifiable information and the models' varying accuracy in different contexts.

Contribution

It uncovers the factors influencing LLMs' association capabilities and demonstrates the privacy implications of their ability to predict PII like emails and phone numbers.

Findings

Model scale correlates with increased association ability.

Lower accuracy in associating PII compared to commonsense knowledge.

LLMs can predict specific PII instances with appropriate prompts.

Abstract

The advancement of large language models (LLMs) brings notable improvements across various applications, while simultaneously raising concerns about potential private data exposure. One notable capability of LLMs is their ability to form associations between different pieces of information, but this raises concerns when it comes to personally identifiable information (PII). This paper delves into the association capabilities of language models, aiming to uncover the factors that influence their proficiency in associating information. Our study reveals that as models scale up, their capacity to associate entities/information intensifies, particularly when target pairs demonstrate shorter co-occurrence distances or higher co-occurrence frequencies. However, there is a distinct performance gap when associating commonsense knowledge versus PII, with the latter showing lower accuracy. Despite the proportion of accurately predicted PII being relatively small, LLMs still demonstrate the capability to predict specific instances of email addresses and phone numbers when provided with appropriate prompts. These findings underscore the potential risk to PII confidentiality posed by the evolving capabilities of LLMs, especially as they continue to expand in scale and power.