Auditing Meta and TikTok Research API Data Access under Article 40(12) of the Digital Services Act

TL;DR

This paper systematically audits the data quality and completeness of Meta and TikTok Research APIs, revealing significant data loss and operational restrictions that hinder independent auditing of platform risks under the Digital Services Act.

Contribution

It provides a quantitative assessment of data gaps and operational limitations in platform Research APIs, highlighting their inadequacy for independent systemic risk analysis.

Findings

Up to 50% data loss due to platform filters

83% metadata stripping from platform data

Research API request limits severely restrict data collection

Abstract

Article 40(12) of the Digital Services Act (DSA) requires Very Large Online Platforms (VLOPs) to provide vetted researchers with access to publicly accessible data. While prior work has identified shortcomings of platform-provided data access mechanisms, existing research has not quantitatively assessed data quality and completeness in Research APIs across platforms, nor systematically mapped how current access provisions fall short. This paper presents a systematic audit of research access modalities by comparing data obtained through platform Research APIs with data collected about the same platforms' user-visible public information environment (PIE). Focusing on two major platform APIs, the TikTok Research API and the Meta Content Library, we reconstruct full information feeds for two controlled sockpuppet accounts during two election periods and benchmark these against the data retrievable for the same posts through the corresponding Research APIs. Our findings show systematic data loss through three classes of platform-imposed mechanisms: scope narrowing, metadata stripping, and operational restrictions. Together, these mechanisms implement overlapping filters that exclude large portions of the platform PIE (up to approximately 50 percent), strip essential contextual metadata (up to approximately 83 percent), and impose severe technical constraints for researchers (down to approximately 1000 requests per day). Viewed through a data quality lens, these filters primarily undermine completeness, resulting in a structurally biased representation of platform activity. We conclude that, in their current form, the Meta and TikTok Research APIs fall short of supporting meaningful, independent auditing of systemic risks as envisioned under the DSA.