How Biased is the Population of Facebook Users? Comparing the Demographics of Facebook Users with Census Data to Generate Correction Factors

TL;DR

This paper assesses demographic biases in Facebook user data by comparing it with official census data, creating correction factors to improve the representativeness of social media-based population estimates.

Contribution

It introduces a methodology to quantify demographic biases in Facebook data using official statistics and provides correction factors for more accurate population analysis.

Findings

High correlation between Facebook data and census for race and political leaning.

Instances of underestimation in official statistics, e.g., immigration counts.

Bias correction factors enable more representative demographic estimates.

Abstract

Censuses around the world are key sources of data to guide government investments and public policies. However, these sources are very expensive to obtain and are collected relatively infrequently. Over the last decade, there has been growing interest in the use of data from social media to complement traditional data sources. However, social media users are not representative of the general population. Thus, analyses based on social media data require statistical adjustments, like post-stratification, in order to remove the bias and make solid statistical claims. These adjustments are possible only when we have information about the frequency of demographic groups using social media. These data, when compared with official statistics, enable researchers to produce appropriate statistical correction factors. In this paper, we leverage the Facebook advertising platform to compile the equivalent of an aggregate-level census of Facebook users. Our compilation includes the population distribution for seven demographic attributes such as gender and age at different geographic levels for the US. By comparing the Facebook counts with official reports provided by the US Census and Gallup, we found very high correlations, especially for political leaning and race. We also identified instances where official statistics may be underestimating population counts as in the case of immigration. We use the information collected to calculate bias correction factors for all computed attributes in order to evaluate the extent to which different demographic groups are more or less represented on Facebook. We provide the first comprehensive analysis for assessing biases in Facebook users across several dimensions. This information can be used to generate bias-adjusted population estimates and demographic counts in a timely way and at fine geographic granularity in between data releases of official statistics