An Empirical Study on Compliance with Ranking Transparency in the Software Documentation of EU Online Platforms

TL;DR

This study evaluates how well six major online platforms comply with EU ranking transparency regulations and introduces automated tools using ChatGPT to assist in compliance assessment, showing promising accuracy.

Contribution

It provides the first empirical analysis of platform compliance with EU transparency rules and develops automated assessment tools based on AI and information retrieval.

Findings

Significant variation in documentation compliance among platforms.

Automated tools show reliable agreement with human judgments.

Potential to improve regulatory oversight and fairness.

Abstract

Compliance with the European Union's Platform-to-Business (P2B) Regulation is challenging for online platforms, and assessing their compliance can be difficult for public authorities. This is partly due to the lack of automated tools for assessing the information (e.g., software documentation) platforms provide concerning ranking transparency. Our study tackles this issue in two ways. First, we empirically evaluate the compliance of six major platforms (Amazon, Bing, Booking, Google, Tripadvisor, and Yahoo), revealing substantial differences in their documentation. Second, we introduce and test automated compliance assessment tools based on ChatGPT and information retrieval technology. These tools are evaluated against human judgments, showing promising results as reliable proxies for compliance assessments. Our findings could help enhance regulatory compliance and align with the United Nations Sustainable Development Goal 10.3, which seeks to reduce inequality, including business disparities, on these platforms.