Beyond the Checkbox: Strengthening DSA Compliance Through Social Media Algorithmic Auditing

TL;DR

This paper critically examines current DSA algorithmic audits, revealing inconsistencies and proposing a behavioral assessment approach to improve compliance verification of AI systems on online platforms.

Contribution

It identifies gaps in current auditing practices and introduces a novel behavioral simulation method to enhance the effectiveness of DSA compliance assessments.

Findings

Current audits lack technical depth and consistency.

Proposed behavioral assessment improves audit robustness.

Highlights need for independent, scalable auditing methods.

Abstract

Algorithms of online platforms are required under the Digital Services Act (DSA) to comply with specific obligations concerning algorithmic transparency, user protection and privacy. To verify compliance with these requirements, DSA mandates platforms to undergo independent audits. Little is known about current auditing practices and their effectiveness in ensuring such compliance. To this end, we bridge regulatory and technical perspectives by critically examining selected audit reports across three critical algorithmic-related provisions: restrictions on profiling minors, transparency in recommender systems, and limitations on targeted advertising using sensitive data. Our analysis shows significant inconsistencies in methodologies and lack of technical depth when evaluating AI-powered systems. To enhance the depth, scale, and independence of compliance assessments, we propose to employ algorithmic auditing -- a process of behavioural assessment of AI algorithms by means of simulating user behaviour, observing algorithm responses and analysing them for audited phenomena.