PrivPRISM: Automatically Detecting Discrepancies Between Google Play Data Safety Declarations and Developer Privacy Policies

TL;DR

PrivPRISM is a framework that automatically detects discrepancies between Google Play's data safety declarations and actual privacy policies, revealing widespread non-compliance and under-disclosure in popular mobile apps.

Contribution

We propose PrivPRISM, a novel language model-based system for scalable, automated comparison of privacy policies and safety declarations to identify inconsistencies.

Findings

53% of popular apps have discrepancies in data disclosures

Privacy policies disclose more sensitive data than safety declarations

Systemic issues like generic policies and vague statements are prevalent

Abstract

End-users seldom read verbose privacy policies, leading app stores like Google Play to mandate simplified data safety declarations as a user-friendly alternative. However, these self-declared disclosures often contradict the full privacy policies, deceiving users about actual data practices and violating regulatory requirements for consistency. To address this, we introduce PrivPRISM, a robust framework that combines encoder and decoder language models to systematically extract and compare fine-grained data practices from privacy policies and to compare against data safety declarations, enabling scalable detection of non-compliance. Evaluating 7,770 popular mobile games uncovers discrepancies in nearly 53% of cases, rising to 61% among 1,711 widely used generic apps. Additionally, static code analysis reveals possible under-disclosures, with privacy policies disclosing just 66.8% of potential accesses to sensitive data like location and financial information, versus only 36.4% in data safety declarations of mobile games. Our findings expose systemic issues, including widespread reuse of generic privacy policies, vague / contradictory statements, and hidden risks in high-profile apps with 100M+ downloads, underscoring the urgent need for automated enforcement to protect platform integrity and for end-users to be vigilant about sensitive data they disclose via popular apps.