Evaluating PDPL Compliance in E-Commerce Websites: Insights and Lessons Learned from Human and LLM Analyses

TL;DR

This study assesses Saudi e-commerce websites' compliance with PDPL, revealing significant gaps, especially among top-ranked sites, and explores using LLMs for automated privacy policy analysis.

Contribution

It provides the first comprehensive compliance assessment of Saudi e-commerce sites and evaluates LLMs as tools for automating privacy policy analysis.

Findings

Only 31% of websites declared all four PDPL items.

Higher non-compliance among top-ranked and locally hosted sites.

LLMs show potential but need improvements for policy analysis.

Abstract

In 2024, Saudi Arabia's Personal Data Protection Law (PDPL) came into force. However, little work has been done to assess its implementation. In this paper, we analyzed 100 e-commerce websites operating in Saudi Arabia against the PDPL, examining the presence of a privacy policy and, if present, the policy's declarations of four items pertaining to personal data rights and practices: 1) personal data retention period, 2) the right to request the destruction of personal data, 3) the right to request a copy of personal data, and 4) a mechanism for filing complaints. Our results show that, despite national awareness and support efforts, a significant fraction of e-commerce websites in our dataset are not fully compliant: only 31% of websites in our dataset declared all four examined items in their privacy policies. Even when privacy policies included such declarations, a considerable fraction of them failed to cover required fine-grained details. Second, the majority of top-ranked e-commerce websites in our dataset (based on search results order) and those hosted on local e-commerce hosting platforms exhibited considerably higher non-compliance rates than mid- to low-ranked websites and those not hosted on local e-commerce platforms. Third, we assessed the use of Large Language Models (LLMs) as an automated tool for privacy policy analysis to measure compliance with the PDPL. We highlight the potential of LLMs and suggest considerations to improve LLM-based automated analysis for privacy policies. Our results provide a step forward in understanding the implementation barriers to data protection laws, especially in non-Western contexts. We provide recommendations for policymakers, regulators, website owners, and developers seeking to improve data protection practices and automate compliance monitoring.