Quality Assessment of Public Summary of Training Content for GPAI models required by AI Act Article 53(1)(d)

TL;DR

This paper develops a framework to evaluate the quality of public summaries of training data for GPAI models, aiming to enhance transparency and stakeholder utility as mandated by the AI Act.

Contribution

It introduces a structured assessment framework for public summaries, enabling comparison, identification of issues, and providing guidelines for improvement.

Findings

Assessment of 5 public summaries reveals varying quality levels.

Framework effectively identifies transparency and usefulness issues.

Provides actionable recommendations for better public summaries.

Abstract

The AI Act's Article 53(1)(d) requires providers of general-purpose AI (GPAI) models to publish a sufficiently detailed public summary about the content used for training based on a template provided by the AI Office. The stated goal of this obligation is to increase transparency regarding the data used for training GPAI models, and to enable relevant stakeholders to exercise their rights, especially regarding IP, copyright, and data protection. This paper provides a quality assessment framework to assess the public summary across two key dimensions: \textit{transparency} regarding information being provided in a clear, comprehensive, and sufficiently detailed manner; and \textit{usefulness} regarding whether the provision of the document and the contents can be effectively utilised by stakeholders to carry out rights related actions. This framework enables identification of key issues in public summaries, and provides a structured and research-based method to compare practices across public summaries and providers. It also enables authorities such as the AI Office to identify potential issues that could emerge and provides actionable recommendations and guidelines for providers to develop public summaries with high quality. The paper provides an assessment of 5 public summaries published as of 12th January 2026 which were found through an exhaustive search process. To disseminate these findings as a public resource, the paper also describes the development of a website where the assessments, outcomes, and methodologies will be shared.