Exploring Large Language Models for Access Control Policy Synthesis and Summarization

TL;DR

This paper evaluates the use of Large Language Models for automatically generating and understanding cloud access control policies, highlighting their strengths and limitations in synthesis and summarization tasks.

Contribution

It introduces a novel semantic-based request summarization method and assesses LLMs' effectiveness in policy synthesis and analysis, revealing their potential and challenges.

Findings

LLMs generate syntactically correct policies with permissiveness issues.

Reasoning LLMs have higher accuracy (93.7%) in policy generation.

LLMs combined with symbolic methods improve policy analysis.

Abstract

Cloud computing is ubiquitous, with a growing number of services being hosted on the cloud every day. Typical cloud compute systems allow administrators to write policies implementing access control rules which specify how access to private data is governed. These policies must be manually written, and due to their complexity can often be error prone. Moreover, existing policies often implement complex access control specifications and thus can be difficult to precisely analyze in determining their behavior works exactly as intended. Recently, Large Language Models (LLMs) have shown great success in automated code synthesis and summarization. Given this success, they could potentially be used for automatically generating access control policies or aid in understanding existing policies. In this paper, we explore the effectiveness of LLMs for access control policy synthesis and summarization. Specifically, we first investigate diverse LLMs for access control policy synthesis, finding that: although LLMs can effectively generate syntactically correct policies, they have permissiveness issues, generating policies equivalent to the given specification 45.8% of the time for non-reasoning LLMs, and 93.7% of the time for reasoning LLMs. We then investigate how LLMs can be used to analyze policies by introducing a novel semantic-based request summarization approach which leverages LLMs to generate a precise characterization of the requests allowed by a policy. Our results show that while there are significant hurdles in leveraging LLMs for automated policy generation, LLMs show promising results when combined with symbolic approaches in analyzing existing policies.