Assessing the Auditability of AI-integrating Systems: A Framework and Learning Analytics Case Study

TL;DR

This paper introduces a comprehensive framework for evaluating the auditability of AI-integrating Learning Analytics systems, emphasizing verifiable claims, evidence, and accessibility to enhance trustworthiness and compliance.

Contribution

It presents a novel framework for assessing auditability in AI-based Learning Analytics systems and demonstrates its application through case studies on Moodle and a prototype system.

Findings

Moodle's auditability is limited by documentation gaps.

Monitoring capabilities are insufficient for effective audits.

Test data availability is a significant challenge.

Abstract

Audits contribute to the trustworthiness of Learning Analytics (LA) systems that integrate Artificial Intelligence (AI) and may be legally required in the future. We argue that the efficacy of an audit depends on the auditability of the audited system. Therefore, systems need to be designed with auditability in mind. We present a framework for assessing the auditability of AI-integrating systems that consists of three parts: (1) Verifiable claims about the validity, utility and ethics of the system, (2) Evidence on subjects (data, models or the system) in different types (documentation, raw sources and logs) to back or refute claims, (3) Evidence must be accessible to auditors via technical means (APIs, monitoring tools, explainable AI, etc.). We apply the framework to assess the auditability of Moodle's dropout prediction system and a prototype AI-based LA. We find that Moodle's auditability is limited by incomplete documentation, insufficient monitoring capabilities and a lack of available test data. The framework supports assessing the auditability of AI-based LA systems in use and improves the design of auditable systems and thus of audits.