HySafe-AI: Hybrid Safety Architectural Analysis Framework for AI Systems: A Case Study

TL;DR

This paper introduces HySAFE-AI, a hybrid safety analysis framework designed to evaluate the safety of complex AI architectures like LLMs and VLMs, improving traditional safety methods for AI systems.

Contribution

The paper presents HySAFE-AI, a novel hybrid safety analysis framework tailored for modern AI architectures, enhancing existing safety evaluation techniques for foundational models.

Findings

Traditional safety analyses can be adapted for AI systems.

HySAFE-AI improves safety evaluation of LLMs and VLMs.

Guidelines for future AI safety standards.

Abstract

AI has become integral to safety-critical areas like autonomous driving systems (ADS) and robotics. The architecture of recent autonomous systems are trending toward end-to-end (E2E) monolithic architectures such as large language models (LLMs) and vision language models (VLMs). In this paper, we review different architectural solutions and then evaluate the efficacy of common safety analyses such as failure modes and effect analysis (FMEA) and fault tree analysis (FTA). We show how these techniques can be improved for the intricate nature of the foundational models, particularly in how they form and utilize latent representations. We introduce HySAFE-AI, Hybrid Safety Architectural Analysis Framework for AI Systems, a hybrid framework that adapts traditional methods to evaluate the safety of AI systems. Lastly, we offer hints of future work and suggestions to guide the evolution of future AI safety standards.