CIMemories: A Compositional Benchmark for Contextual Integrity of Persistent Memory in LLMs

TL;DR

CIMemories introduces a benchmark to evaluate how well large language models control sensitive information flow from persistent memory based on context, revealing significant privacy violations and instability in current models.

Contribution

The paper presents CIMemories, a novel benchmark for assessing contextual integrity in LLMs' memory management, highlighting critical privacy risks and model limitations.

Findings

Frontier models leak up to 69% of attributes inappropriately.

Violation rates increase with task repetitions and usage.

Models tend to overgeneralize, sharing all or nothing.

Abstract

Large Language Models (LLMs) increasingly use persistent memory from past interactions to enhance personalization and task performance. However, this memory introduces critical risks when sensitive information is revealed in inappropriate contexts. We present CIMemories, a benchmark for evaluating whether LLMs appropriately control information flow from memory based on task context. CIMemories uses synthetic user profiles with over 100 attributes per user, paired with diverse task contexts in which each attribute may be essential for some tasks but inappropriate for others. Our evaluation reveals that frontier models exhibit up to 69% attribute-level violations (leaking information inappropriately), with lower violation rates often coming at the cost of task utility. Violations accumulate across both tasks and runs: as usage increases from 1 to 40 tasks, GPT-5's violations rise from 0.1% to 9.6%, reaching 25.1% when the same prompt is executed 5 times, revealing arbitrary and unstable behavior in which models leak different attributes for identical prompts. Privacy-conscious prompting does not solve this - models overgeneralize, sharing everything or nothing rather than making nuanced, context-dependent decisions. These findings reveal fundamental limitations that require contextually aware reasoning capabilities, not just better prompting or scaling.