AIMS.au: A Dataset for the Analysis of Modern Slavery Countermeasures in Corporate Statements

TL;DR

This paper introduces AIMS.au, a detailed dataset of 5,731 corporate statements on modern slavery, enabling improved detection of concrete countermeasures using machine learning, and benchmarks language models for compliance assessment.

Contribution

The paper presents a new annotated dataset for analyzing corporate modern slavery statements and demonstrates its use in benchmarking language models for compliance detection.

Findings

Effective dataset construction for sentence-level annotation.

Language models can identify relevant slavery countermeasure statements.

Benchmark results show potential for automated compliance assessment.

Abstract

Despite over a decade of legislative efforts to address modern slavery in the supply chains of large corporations, the effectiveness of government oversight remains hampered by the challenge of scrutinizing thousands of statements annually. While Large Language Models (LLMs) can be considered a well established solution for the automatic analysis and summarization of documents, recognizing concrete modern slavery countermeasures taken by companies and differentiating those from vague claims remains a challenging task. To help evaluate and fine-tune LLMs for the assessment of corporate statements, we introduce a dataset composed of 5,731 modern slavery statements taken from the Australian Modern Slavery Register and annotated at the sentence level. This paper details the construction steps for the dataset that include the careful design of annotation specifications, the selection and preprocessing of statements, and the creation of high-quality annotation subsets for effective model evaluations. To demonstrate our dataset's utility, we propose a machine learning methodology for the detection of sentences relevant to mandatory reporting requirements set by the Australian Modern Slavery Act. We then follow this methodology to benchmark modern language models under zero-shot and supervised learning settings.