Longitudinal Compliance Analysis of Android Applications with Privacy Policies

TL;DR

This study longitudinally analyzes 268 Android apps from 2008 to 2016, revealing increasing non-compliance between privacy policies and actual data practices, highlighting privacy transparency issues over time.

Contribution

It introduces a machine learning-based method to detect discrepancies between privacy policies and app behaviors across multiple versions.

Findings

Increase in undisclosed data collection practices over time

Newer app versions tend to be more non-compliant

Privacy policies often do not match actual app behaviors

Abstract

Contemporary mobile applications (apps) are designed to track, use, and share users' data, often without their consent, which results in potential privacy and transparency issues. To investigate whether mobile apps have always been (non-)transparent regarding how they collect information about users, we perform a longitudinal analysis of the historical versions of 268 Android apps. These apps comprise 5,240 app releases or versions between 2008 and 2016. We detect inconsistencies between apps' behaviors and the stated use of data collection in privacy policies to reveal compliance issues. We utilize machine learning techniques for the classification of the privacy policy text to identify the purported practices that collect and/or share users' personal information, such as phone numbers and email addresses. We then uncover the data leaks of an app through static and dynamic analysis. Over time, our results show a steady increase in the number of apps' data collection practices that are undisclosed in the privacy policies. This behavior is particularly troubling since privacy policy is the primary tool for describing the app's privacy protection practices. We find that newer versions of the apps are likely to be more non-compliant than their preceding versions. The discrepancies between the purported and the actual data practices show that privacy policies are often incoherent with the apps' behaviors, thus defying the 'notice and choice' principle when users install apps.