SETA: Statistical Fault Attribution for Compound AI Systems

TL;DR

This paper introduces SETA, a modular robustness testing framework for complex AI systems with multiple neural networks, enabling detailed error analysis and propagation understanding across components.

Contribution

SETA is a novel, architecture-agnostic framework that allows component-wise robustness testing and error propagation analysis in multi-network AI systems.

Findings

Effective error isolation in multi-network systems

Demonstrated fine-grained robustness analysis on autonomous rail inspection

Scalable approach applicable across domains

Abstract

Modern AI systems increasingly comprise multiple interconnected neural networks to tackle complex inference tasks. Testing such systems for robustness and safety entails significant challenges. Current state-of-the-art robustness testing techniques, whether black-box or white-box, have been proposed and implemented for single-network models and do not scale well to multi-network pipelines. We propose a modular robustness testing framework that applies a given set of perturbations to test data. Our testing framework supports (1) a component-wise system analysis to isolate errors and (2) reasoning about error propagation across the neural network modules. The testing framework is architecture and modality agnostic and can be applied across domains. We apply the framework to a real-world autonomous rail inspection system composed of multiple deep networks and successfully demonstrate how our approach enables fine-grained robustness analysis beyond conventional end-to-end metrics.