Personalizing Agent Privacy Decisions via Logical Entailment

TL;DR

This paper introduces ARIEL, a framework combining large language models with rule-based logic to improve personalized privacy decision-making, achieving significantly higher accuracy than traditional in-context learning methods.

Contribution

ARIEL is a novel approach that integrates logical entailment with LLMs to enable personalized and interpretable privacy reasoning for user data sharing decisions.

Findings

ARIEL reduces F1 error rate by 40.6% compared to standard ICL methods.

The framework effectively captures individual privacy preferences.

ARIEL improves the accuracy of privacy judgments in experimental evaluations.

Abstract

Personal large language model (LLM) agents increasingly perform tasks that require access to user data, raising concerns about appropriate data disclosure. We show that relying solely on LLMs to make data-sharing decisions is insufficient. Prompting LLMs with general privacy norms fails to capture individual users' privacy preferences, while providing prior user data-sharing decisions through in-context learning (ICL) leads to unreliable and opaque reasoning. To address these limitations, we propose ARIEL (Agentic Reasoning with Individualized Entailment Logic), a framework that combines LLMs with rule-based logic to enable structured, personalized privacy reasoning. The core mechanism of ARIEL determines whether a user's prior decision on a data-sharing request $\textit{logically entails}$ the same decision for a new request. Experimental evaluations using advanced models and public datasets show that ARIEL reduces the F1 error rate for appropriate judgments by $\textbf{40.6%}$ compared to standard ICL-based reasoning, indicating that ARIEL is effective at correctly judging requests where the user would approve data sharing. These results demonstrate that integrating LLMs with logical entailment provides an effective and interpretable approach for automating personalized privacy decisions.