The Missing Variable: Socio-Technical Alignment in Risk Evaluation

TL;DR

This paper introduces a socio-technical alignment variable to improve risk evaluation of AI-enabled safety-critical systems by accounting for human, technical, and organizational interactions.

Contribution

It proposes a novel socio-technical alignment variable integrated into risk assessment, addressing a key gap in current methods for AI-enabled systems.

Findings

The $STA$ variable captures socio-technical interactions affecting safety.

In a case study, $STA$ distinguishes between naive and safeguarded system designs.

Traditional risk methods overlook critical socio-technical safety factors.

Abstract

This paper addresses a critical gap in the risk assessment of AI-enabled safety-critical systems. While these systems, where AI systems assists human operators, function as complex socio-technical systems, existing risk evaluation methods fail to account for the associated complex interaction between human, technical, and organizational elements. Through a comparative analysis of system attributes from both socio-technical and AI-enabled systems and a review of current risk evaluation methods, we confirm the absence of socio-technical considerations in standard risk expressions. To bridge this gap, we introduce a novel socio-technical alignment $STA$ variable designed to be integrated into the foundational risk equation. This variable estimates the degree of harmonious interaction between the AI systems, human operators, and organizational processes. A case study on an AI-enabled liquid hydrogen bunkering system demonstrates the variable's relevance. By comparing a naive and a safeguarded system design, we illustrate how the $STA$-augmented expression captures socio-technical safety implications that traditional risk evaluation overlooks, providing a more holistic basis for risk evaluation.