Entailment-Driven Privacy Policy Classification with LLMs

TL;DR

This paper introduces an entailment-driven LLM framework that classifies privacy policy paragraphs into user-friendly labels, outperforming traditional methods and offering explainability to help users understand data collection practices.

Contribution

The paper presents a novel entailment-based approach using LLMs for classifying privacy policy content with improved accuracy and interpretability.

Findings

F1 score improved by 11.2% over traditional LLM methods

Framework provides inherently explainable predictions

Enhances user understanding of privacy policies

Abstract

While many online services provide privacy policies for end users to read and understand what personal data are being collected, these documents are often lengthy and complicated. As a result, the vast majority of users do not read them at all, leading to data collection under uninformed consent. Several attempts have been made to make privacy policies more user friendly by summarising them, providing automatic annotations or labels for key sections, or by offering chat interfaces to ask specific questions. With recent advances in Large Language Models (LLMs), there is an opportunity to develop more effective tools to parse privacy policies and help users make informed decisions. In this paper, we propose an entailment-driven LLM based framework to classify paragraphs of privacy policies into meaningful labels that are easily understood by users. The results demonstrate that our framework outperforms traditional LLM methods, improving the F1 score in average by 11.2%. Additionally, our framework provides inherently explainable and meaningful predictions.