The Algorithmic Self-Portrait: Deconstructing Memory in ChatGPT

TL;DR

This paper investigates how memory in ChatGPT creates personalized user profiles, analyzing real-world data to reveal privacy concerns and proposing a framework to enhance user control and data protection.

Contribution

It provides the first large-scale analysis of ChatGPT's memory creation process and introduces Attribution Shield, a framework to improve privacy and user agency.

Findings

96% of memories are created unilaterally by the system

28% of memories contain GDPR-defined personal data

52% of memories include psychological insights

Abstract

To enable personalized and context-aware interactions, conversational AI systems have introduced a new mechanism: Memory. Memory creates what we refer to as the Algorithmic Self-portrait - a new form of personalization derived from users' self-disclosed information divulged within private conversations. While memory enables more coherent exchanges, the underlying processes of memory creation remain opaque, raising critical questions about data sensitivity, user agency, and the fidelity of the resulting portrait. To bridge this research gap, we analyze 2,050 memory entries from 80 real-world ChatGPT users. Our analyses reveal three key findings: (1) A striking 96% of memories in our dataset are created unilaterally by the conversational system, potentially shifting agency away from the user; (2) Memories, in our dataset, contain a rich mix of GDPR-defined personal data (in 28% memories) along with psychological insights about participants (in 52% memories); and (3)~A significant majority of the memories (84%) are directly grounded in user context, indicating faithful representation of the conversations. Finally, we introduce a framework-Attribution Shield-that anticipates these inferences, alerts about potentially sensitive memory inferences, and suggests query reformulations to protect personal information without sacrificing utility.