Can LLMs Make (Personalized) Access Control Decisions?

TL;DR

This paper explores using large language models to make personalized, context-aware access control decisions, aiming to reduce user burden while aligning with individual privacy preferences.

Contribution

It introduces a dataset of user privacy preferences and permission decisions, and evaluates LLMs' ability to replicate and improve these decisions in a smartphone permission context.

Findings

LLMs agree with user decisions up to 86% of the time.

Personalized LLMs better reflect individual preferences.

Strict adherence to preferences may lead to over-permission and less safety.

Abstract

Precise access control decisions are crucial for the security of both traditional applications and emerging agent-based systems. Typically, these decisions are made by users during app installation or at runtime. However, due to the increasing complexity and automation of systems, making access control decisions can impose a significant cognitive burden on users, often overwhelming them and leading to suboptimal or even arbitrary choices. To address this problem, we investigate the ability of LLMs to make dynamic, context-aware decisions aligned with users' security preferences, expressed during a lightweight setup phase. As a case study, we analyze smartphone application permission requests, given their ubiquity and users' familiarity with them. We curated a dataset comprising 307 user privacy statements (short, natural-language descriptions of user preferences) and 14,682 corresponding permission decisions, gathered from smartphone users in an online data collection. We compare these decisions with those made by two versions of LLMs that are tasked with reasoning about the app and the request context: a general model and a personalized one (which incorporates user preferences). For the latter, we also collected user feedback on 1,298 of its decisions. Our results show that LLMs generally reflect users' preferences well, agreeing with the majority decision in up to 86% of cases, and can steer users toward safer behavior. However, the results also reveal a key trade-off in personalization: while incorporating user-specific privacy preferences improves agreement with individual decisions, strict adherence to these preferences may lead to less safe outcomes, as users tend to over-permission.