Autonomous Systems Dependability in the era of AI: Design Challenges in Safety, Security, Reliability and Certification

TL;DR

This paper discusses the challenges and emerging solutions for ensuring dependability in autonomous safety-critical systems integrating AI and ML, focusing on design, verification, and certification.

Contribution

It highlights new methodologies and frameworks for designing, verifying, and certifying AI-enabled autonomous systems to address their inherent uncertainties and non-determinism.

Findings

Advances in reliability modeling for AI components

Frameworks for secure system design with AI/ML

Certification approaches for learning-enabled systems

Abstract

The design of embedded safety-critical systems such as those used in next-generation automotive and autonomous platforms, is increasingly challenged by escalating system complexity, hardware-software heterogeneity, and the integration of intelligent, data-driven components. Ensuring dependability in such systems requires a holistic approach that spans multiple abstraction layers and encompasses both design- and run-time assurance. Traditional methods for reliability, safety, and security management often fall short in addressing the dynamic and uncertain behaviors introduced by Artificial Intelligence (AI) and Machine Learning (ML) components, especially under stringent real-time, power, and safety constraints. While AI and ML offer powerful predictive, adaptive, and self-optimizing capabilities that can enhance system dependability, their inherent non-determinism, data-dependence, and lack of formal guarantees introduce new challenges for verification, validation, and certification. This paper explores emerging methodologies, architectures, and frameworks for designing dependable autonomous and embedded systems in the era of AI. It highlight advances in reliability modeling, secure system design, and certification approaches that account for imperfect, learning-enabled components, aiming to bridge the gap between AI innovation and certifiable system-level dependability.