Doing Audits Right? The Role of Sampling and Legal Content Analysis in Systemic Risk Assessments and Independent Audits in the Digital Services Act

TL;DR

This paper evaluates methods for systemic risk audits under the EU's Digital Services Act, emphasizing sampling and legal analysis to improve evidence-based assessments of illegal content and other risks on online platforms.

Contribution

It proposes a structured, mixed-method approach combining sampling and legal analysis for effective DSA compliance audits, addressing methodological limitations.

Findings

Sampling combined with legal analysis enhances risk detection.

Initial audit reports reveal diverse sampling methodologies.

Structured approaches improve audit reliability and compliance.

Abstract

A central requirement of the European Union's Digital Services Act (DSA) is that online platforms undergo internal and external audits. A key component of these audits is the assessment of systemic risks, including the dissemination of illegal content, threats to fundamental rights, impacts on democratic processes, and gender-based violence. The DSA Delegated Regulation outlines how such audits should be conducted, setting expectations for both platforms and auditors. This article evaluates the strengths and limitations of different qualitative and quantitative methods for auditing these systemic risks and proposes a mixed-method approach for DSA compliance. We argue that content sampling, combined with legal and empirical analysis, offers a viable method for risk-specific audits. First, we examine relevant legal provisions on sample selection for audit purposes. We then assess sampling techniques and methods suitable for detecting systemic risks, focusing on how representativeness can be understood across disciplines. Finally, we review initial systemic risk assessment reports submitted by platforms, analyzing their testing and sampling methodologies. By proposing a structured, mixed-method approach tailored to specific risk categories and platform characteristics, this article addresses the challenge of evidence-based audits under the DSA. Our contribution emphasizes the need for adaptable, context-sensitive auditing strategies and adds to the emerging field of DSA compliance research.