A Decision-driven Methodology for Designing Uncertainty-aware AI Self-Assessment

TL;DR

This paper presents a decision-driven methodology for designing uncertainty-aware AI self-assessment tools, aiming to improve trustworthiness and decision-making in high-impact scenarios by categorizing methods and providing practical guidelines.

Contribution

It introduces a comprehensive, practical framework for selecting and designing AI self-assessment methods considering decision impact and real-world application needs.

Findings

Categorized AI self-assessment methods along key dimensions.

Provided guidelines for method selection based on decision impact.

Demonstrated utility through two national-interest scenarios.

Abstract

Artificial intelligence (AI) has revolutionized decision-making processes and systems throughout society and, in particular, has emerged as a significant technology in high-impact scenarios of national interest. Yet, despite AI's impressive predictive capabilities in controlled settings, it still suffers from a range of practical setbacks preventing its widespread use in various critical scenarios. In particular, it is generally unclear if a given AI system's predictions can be trusted by decision-makers in downstream applications. To address the need for more transparent, robust, and trustworthy AI systems, a suite of tools has been developed to quantify the uncertainty of AI predictions and, more generally, enable AI to "self-assess" the reliability of its predictions. In this manuscript, we categorize methods for AI self-assessment along several key dimensions and provide guidelines for selecting and designing the appropriate method for a practitioner's needs. In particular, we focus on uncertainty estimation techniques that consider the impact of self-assessment on the choices made by downstream decision-makers and on the resulting costs and benefits of decision outcomes. To demonstrate the utility of our methodology for self-assessment design, we illustrate its use for two realistic national-interest scenarios. This manuscript is a practical guide for machine learning engineers and AI system users to select the ideal self-assessment techniques for each problem.