Lalaine: Measuring and Characterizing Non-Compliance of Apple Privacy Labels at Scale

TL;DR

This study introduces Lalaine, a new methodology to systematically evaluate the accuracy and compliance of Apple privacy labels against actual app data practices, revealing significant non-compliance issues.

Contribution

Lalaine is the first systematic approach to measure privacy-label accuracy at scale, analyzing over 5,000 iOS apps to uncover privacy-label non-compliance and root causes.

Findings

High prevalence of privacy-label non-compliance

Identified root causes for inaccurate privacy labels

Insights for improving privacy-label design and enforcement

Abstract

As a key supplement to privacy policies that are known to be lengthy and difficult to read, Apple has launched the app privacy labels, which purportedly help users more easily understand an app's privacy practices. However, false and misleading privacy labels can dupe privacy-conscious consumers into downloading data-intensive apps, ultimately eroding the credibility and integrity of the labels. Although Apple releases requirements and guidelines for app developers to create privacy labels, little is known about whether and to what extent the privacy labels in the wild are correct and compliant, reflecting the actual data practices of iOS apps. This paper presents the first systematic study, based on our new methodology named Lalaine, to evaluate data-flow to privacy-label (flow-to-label) consistency. Lalaine analyzed the privacy labels and binaries of 5,102 iOS apps, shedding light on the prevalence and seriousness of privacy-label non-compliance. We provide detailed case studies and analyze root causes for privacy label non-compliance that complements prior understandings. This has led to new insights for improving privacy-label design and compliance requirements, so app developers, platform stakeholders, and policy-makers can better achieve their privacy and accountability goals. Lalaine is thoroughly evaluated for its high effectiveness and efficiency. We are responsibly reporting the results to stakeholders.