Logging Requirement for Continuous Auditing of Responsible Machine Learning-based Applications

TL;DR

This paper explores how logging practices can be enhanced to enable continuous auditing of machine learning systems, addressing transparency, fairness, and accountability concerns in responsible AI deployment.

Contribution

It identifies deficiencies in current logging practices for ML and proposes improved methods and tooling to support responsible AI auditing and compliance.

Findings

Need for systematic logging of responsible AI metrics

Enhanced logging supports transparency and accountability

Guidance for practitioners and tool developers

Abstract

Machine learning (ML) is increasingly applied across industries to automate decision-making, but concerns about ethical and legal compliance remain due to limited transparency, fairness, and accountability. Monitoring through logging a long-standing practice in traditional software offers a potential means for auditing ML applications, as logs provide traceable records of system behavior useful for debugging, performance analysis, and continuous auditing. systematically auditing models for compliance or accountability. The findings underscore the need for enhanced logging practices and tooling that systematically integrate responsible AI metrics. Such practices would support the development of auditable, transparent, and ethically responsible ML systems, aligning with growing regulatory requirements and societal expectations. By highlighting specific deficiencies and opportunities, this work provides actionable guidance for both practitioners and tool developers seeking to strengthen the accountability and trustworthiness of ML applications.