A Survey on AI Assurance

TL;DR

This survey reviews AI assurance research from 1985 to 2021, introducing a new definition, a comparison framework, and a scoring system to evaluate assurance methods, aiming to clarify and guide future developments.

Contribution

It provides a structured overview of AI assurance research, proposing a new definition, a comparative framework, and a scoring system to evaluate existing methods.

Findings

Developed a ten-metric scoring system for AI assurance methods

Contrasted and tabulated various assurance approaches

Provided a roadmap and future directions for AI assurance

Abstract

Artificial Intelligence (AI) algorithms are increasingly providing decision making and operational support across multiple domains. AI includes a wide library of algorithms for different problems. One important notion for the adoption of AI algorithms into operational decision process is the concept of assurance. The literature on assurance, unfortunately, conceals its outcomes within a tangled landscape of conflicting approaches, driven by contradicting motivations, assumptions, and intuitions. Accordingly, albeit a rising and novel area, this manuscript provides a systematic review of research works that are relevant to AI assurance, between years 1985 - 2021, and aims to provide a structured alternative to the landscape. A new AI assurance definition is adopted and presented and assurance methods are contrasted and tabulated. Additionally, a ten-metric scoring system is developed and introduced to evaluate and compare existing methods. Lastly, in this manuscript, we provide foundational insights, discussions, future directions, a roadmap, and applicable recommendations for the development and deployment of AI assurance.