Auditing Work: Exploring the New York City algorithmic bias audit regime

TL;DR

This paper critically examines New York City's first algorithmic bias audit law, revealing its shortcomings and offering recommendations to improve future bias auditing regimes for automated hiring tools.

Contribution

It provides an in-depth qualitative analysis of LL 144, highlighting key challenges and proposing actionable policy recommendations for effective algorithmic bias auditing.

Findings

LL 144 lacks clear definitions for key terms like AEDTs and auditors.

Industry lobbying and practical challenges hinder effective audits.

Four distinct auditor roles emerge, indicating ambiguity in implementation.

Abstract

In July 2023, New York City (NYC) initiated the first algorithm auditing system for commercial machine-learning systems. Local Law 144 (LL 144) mandates NYC-based employers using automated employment decision-making tools (AEDTs) in hiring to undergo annual bias audits conducted by an independent auditor. This paper examines lessons from LL 144 for other national algorithm auditing attempts. Through qualitative interviews with 16 experts and practitioners within the regime, we find that LL 144 has not effectively established an auditing regime. The law fails to clearly define key aspects, such as AEDTs and independent auditors, leading auditors, AEDT vendors, and companies using AEDTs to define the law's practical implementation in ways that failed to protect job applicants. Contributing factors include the law's flawed transparency-driven theory of change, industry lobbying narrowing the definition of AEDTs, practical and cultural challenges faced by auditors in accessing data, and wide disagreement over what constitutes a legitimate auditor, resulting in four distinct 'auditor roles.' We conclude with four recommendations for policymakers seeking to create similar bias auditing regimes, emphasizing clearer definitions, metrics, and increased accountability. By exploring LL 144 through the lens of auditors, our paper advances the evidence base around audit as an accountability mechanism, providing guidance for policymakers seeking to create similar regimes.