GoldCoin: Grounding Large Language Models in Privacy Laws via Contextual Integrity Theory

TL;DR

GoldCoin is a novel framework that uses contextual integrity theory to ground large language models in privacy laws, improving their ability to identify privacy violations in complex social contexts and real court cases.

Contribution

It introduces a new approach leveraging synthetic scenarios based on privacy laws to enhance LLM understanding of privacy violations within social contexts.

Findings

GoldCoin significantly improves LLM performance in recognizing privacy risks.

The framework surpasses baseline models on judicial privacy tasks.

Synthetic scenario generation aids in legal comprehension of privacy issues.

Abstract

Privacy issues arise prominently during the inappropriate transmission of information between entities. Existing research primarily studies privacy by exploring various privacy attacks, defenses, and evaluations within narrowly predefined patterns, while neglecting that privacy is not an isolated, context-free concept limited to traditionally sensitive data (e.g., social security numbers), but intertwined with intricate social contexts that complicate the identification and analysis of potential privacy violations. The advent of Large Language Models (LLMs) offers unprecedented opportunities for incorporating the nuanced scenarios outlined in privacy laws to tackle these complex privacy issues. However, the scarcity of open-source relevant case studies restricts the efficiency of LLMs in aligning with specific legal statutes. To address this challenge, we introduce a novel framework, GoldCoin, designed to efficiently ground LLMs in privacy laws for judicial assessing privacy violations. Our framework leverages the theory of contextual integrity as a bridge, creating numerous synthetic scenarios grounded in relevant privacy statutes (e.g., HIPAA), to assist LLMs in comprehending the complex contexts for identifying privacy risks in the real world. Extensive experimental results demonstrate that GoldCoin markedly enhances LLMs' capabilities in recognizing privacy risks across real court cases, surpassing the baselines on different judicial tasks.