Judging by the Rules: Compliance-Aligned Framework for Modern Slavery Statement Monitoring

TL;DR

This paper introduces a rule-based AI framework for monitoring modern slavery statements, ensuring compliance assessments are transparent, verifiable, and aligned with legal standards, addressing limitations of existing LLM applications.

Contribution

It presents the Compliance Alignment Judge and CALLM, novel AI components that produce rule-consistent, transparent outputs for legal compliance verification of slavery disclosures.

Findings

CALLM improves compliance prediction accuracy

Outputs are transparent and legally grounded

Framework enables expert oversight and verification

Abstract

Modern slavery affects millions of people worldwide, and regulatory frameworks such as Modern Slavery Acts now require companies to publish detailed disclosures. However, these statements are often vague and inconsistent, making manual review time-consuming and difficult to scale. While NLP offers a promising path forward, high-stakes compliance tasks require more than accurate classification: they demand transparent, rule-aligned outputs that legal experts can verify. Existing applications of large language models (LLMs) often reduce complex regulatory assessments to binary decisions, lacking the necessary structure for robust legal scrutiny. We argue that compliance verification is fundamentally a rule-matching problem: it requires evaluating whether textual statements adhere to well-defined regulatory rules. To this end, we propose a novel framework that harnesses AI for rule-level compliance verification while preserving expert oversight. At its core is the Compliance Alignment Judge (CA-Judge), which evaluates model-generated justifications based on their fidelity to statutory requirements. Using this feedback, we train the Compliance Alignment LLM (CALLM), a model that produces rule-consistent, human-verifiable outputs. CALLM improves predictive performance and generates outputs that are both transparent and legally grounded, offering a more verifiable and actionable solution for real-world compliance analysis.