ConsentDiff at Scale: Longitudinal Audits of Web Privacy Policy Changes and UI Frictions

TL;DR

ConsentDiff offers a novel longitudinal analysis of web privacy policies and consent interfaces, tracking changes over time and assessing UI friction and policy consistency.

Contribution

It introduces a reproducible pipeline for tracking policy and UI changes, including a new alignment score connecting policy claims to UI cues.

Findings

Web policies show ongoing churn and systematic UI design changes.

Higher claim-UI alignment occurs where rejecting consent is visible.

Lower friction interfaces are associated with better claim-UI alignment.

Abstract

Web privacy is experienced via two public artifacts: site utterances in policy texts, and the actions users are required to take during consent interfaces. In the extensive cross-section audits we've studied, there is a lack of longitudinal data detailing how these artifacts are changing together, and if interfaces are actually doing what they promise in policy. ConsentDiff provides that longitudinal view. We build a reproducible pipeline that snapshots sites every month, semantically aligns policy clauses to track clause-level churn, and classifies consent-UI patterns by pulling together DOM signals with cues provided by screenshots. We introduce a novel weighted claim-UI alignment score, connecting common policy claims to observable predicates, and enabling comparisons over time, regions, and verticals. Our measurements suggest continued policy churn, systematic changes to eliminate a higher-friction banner design, and significantly higher alignment where rejecting is visible and lower friction.