A Holistic Assessment of the Reliability of Machine Learning Systems

TL;DR

This paper introduces a comprehensive framework for assessing the reliability of machine learning systems across multiple key properties, providing a holistic view of their robustness in high-stakes applications.

Contribution

It proposes a new holistic assessment methodology and reliability score, evaluating multiple reliability metrics and analyzing over 500 models to identify techniques that improve overall system dependability.

Findings

Designing for one metric does not limit others

Certain algorithms can enhance multiple reliability aspects simultaneously

The framework offers a comprehensive reliability evaluation approach

Abstract

As machine learning (ML) systems increasingly permeate high-stakes settings such as healthcare, transportation, military, and national security, concerns regarding their reliability have emerged. Despite notable progress, the performance of these systems can significantly diminish due to adversarial attacks or environmental changes, leading to overconfident predictions, failures to detect input faults, and an inability to generalize in unexpected scenarios. This paper proposes a holistic assessment methodology for the reliability of ML systems. Our framework evaluates five key properties: in-distribution accuracy, distribution-shift robustness, adversarial robustness, calibration, and out-of-distribution detection. A reliability score is also introduced and used to assess the overall system reliability. To provide insights into the performance of different algorithmic approaches, we identify and categorize state-of-the-art techniques, then evaluate a selection on real-world tasks using our proposed reliability metrics and reliability score. Our analysis of over 500 models reveals that designing for one metric does not necessarily constrain others but certain algorithmic techniques can improve reliability across multiple metrics simultaneously. This study contributes to a more comprehensive understanding of ML reliability and provides a roadmap for future research and development.