Honesty is the Best Policy: On the Accuracy of Apple Privacy Labels Compared to Apps' Privacy Policies
Mir Masood Ali, David G. Balash, Monica Kodwani, Chris Kanich, and Adam J. Aviv

TL;DR
This study compares Apple’s privacy labels with the actual privacy policies of nearly half a million apps, revealing significant discrepancies and suggesting improvements for more accurate privacy labeling.
Contribution
We developed a BERT-based model to automatically extract privacy policy features and identify inconsistencies with Apple’s privacy labels at scale.
Findings
Many apps report less data collection than policies indicate
Most apps with 'Data Not Collected' labels have conflicting policies
Discrepancies often stem from template use and unclear definitions
Abstract
Apple introduced privacy labels in Dec. 2020 as a way for developers to report the privacy behaviors of their apps. While Apple does not validate labels, they also require developers to provide a privacy policy, which offers an important comparison point. In this paper, we fine-tuned BERT-based language models to extract privacy policy features for 474,669 apps on the iOS App Store, comparing the output to the privacy labels. We identify discrepancies between the policies and the labels, particularly as they relate to data collected linked to users. We find that 228K apps' privacy policies may indicate data collection linked to users than what is reported in the privacy labels. More alarming, a large number (97%) of the apps with a Data Not Collected privacy label have a privacy policy indicating otherwise. We provide insights into potential sources for discrepancies, including the use…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsPrivacy, Security, and Data Protection · Sexuality, Behavior, and Technology · Advanced Malware Detection Techniques
