A Computationally Efficient Learning of Artificial Intelligence System Reliability Considering Error Propagation

TL;DR

This paper introduces a new, efficient framework for modeling AI system reliability that explicitly accounts for error propagation across stages, using simulation data and a composite likelihood EM algorithm, demonstrated on autonomous vehicle perception systems.

Contribution

It develops a novel reliability modeling approach that explicitly captures error propagation, addressing data scarcity, interdependence, and computational challenges in AI systems.

Findings

The framework accurately predicts system reliability.

It demonstrates computational efficiency in complex error analysis.

Application to autonomous vehicles shows practical effectiveness.

Abstract

Artificial Intelligence (AI) systems are increasingly prominent in emerging smart cities, yet their reliability remains a critical concern. These systems typically operate through a sequence of interconnected functional stages, where upstream errors may propagate to downstream stages, ultimately affecting overall system reliability. Quantifying such error propagation is essential for accurate modeling of AI system reliability. However, this task is challenging due to: i) data availability: real-world AI system reliability data are often scarce and constrained by privacy concerns; ii) model validity: recurring error events across sequential stages are interdependent, violating the independence assumptions of statistical inference; and iii) computational complexity: AI systems process large volumes of high-speed data, resulting in frequent and complex recurrent error events that are difficult to track and analyze. To address these challenges, this paper leverages a physics-based autonomous vehicle simulation platform with a justifiable error injector to generate high-quality data for AI system reliability analysis. Building on this data, a new reliability modeling framework is developed to explicitly characterize error propagation across stages. Model parameters are estimated using a computationally efficient, theoretically guaranteed composite likelihood expectation - maximization algorithm. Its application to the reliability modeling for autonomous vehicle perception systems demonstrates its predictive accuracy and computational efficiency.