Automated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discrimination

TL;DR

This paper introduces AdFisher, a tool that automates experiments to analyze how Google's ad settings influence user profiling, revealing opacity, limited choice, and potential discrimination in ad delivery.

Contribution

We developed AdFisher, an automated experimental framework combining machine learning and statistical analysis to study ad personalization and discrimination.

Findings

Ad Settings is opaque about user profile features.

Users have limited control over ad personalization.

Discrimination observed based on gender and webpage visited.

Abstract

To partly address people's concerns over web tracking, Google has created the Ad Settings webpage to provide information about and some choice over the profiles Google creates on users. We present AdFisher, an automated tool that explores how user behaviors, Google's ads, and Ad Settings interact. AdFisher can run browser-based experiments and analyze data using machine learning and significance tests. Our tool uses a rigorous experimental design and statistical analysis to ensure the statistical soundness of our results. We use AdFisher to find that the Ad Settings was opaque about some features of a user's profile, that it does provide some choice on ads, and that these choices can lead to seemingly discriminatory ads. In particular, we found that visiting webpages associated with substance abuse changed the ads shown but not the settings page. We also found that setting the gender to female resulted in getting fewer instances of an ad related to high paying jobs than setting it to male. We cannot determine who caused these findings due to our limited visibility into the ad ecosystem, which includes Google, advertisers, websites, and users. Nevertheless, these results can form the starting point for deeper investigations by either the companies themselves or by regulatory bodies.