Evolving AI Risk Management: A Maturity Model based on the NIST AI Risk Management Framework

TL;DR

This paper introduces a flexible maturity model based on the NIST AI RMF to evaluate and improve organizational practices in sociotechnical harm mitigation, addressing gaps in implementation and effectiveness.

Contribution

It develops a novel maturity model grounded in the NIST AI RMF to assess and guide organizations in operationalizing AI risk management practices.

Findings

Organizations currently lag in implementing AI risk management practices.

The maturity model provides a structured pathway for improvement.

Framework helps align organizational practices with emerging AI risk standards.

Abstract

Researchers, government bodies, and organizations have been repeatedly calling for a shift in the responsible AI community from general principles to tangible and operationalizable practices in mitigating the potential sociotechnical harms of AI. Frameworks like the NIST AI RMF embody an emerging consensus on recommended practices in operationalizing sociotechnical harm mitigation. However, private sector organizations currently lag far behind this emerging consensus. Implementation is sporadic and selective at best. At worst, it is ineffective and can risk serving as a misleading veneer of trustworthy processes, providing an appearance of legitimacy to substantively harmful practices. In this paper, we provide a foundation for a framework for evaluating where organizations sit relative to the emerging consensus on sociotechnical harm mitigation best practices: a flexible maturity model based on the NIST AI RMF.