Alert of the Second Decision-maker: An Introduction to Human-AI Conflict

TL;DR

This paper introduces the concept of human-AI conflict, highlighting its causes, measurement, and risks, emphasizing that AI can act as a second decision-maker and pose unique challenges in digital systems.

Contribution

It systematically defines human-AI conflict, distinguishes it from traditional faults, and emphasizes its importance as an emerging interdisciplinary risk in digitalized processes.

Findings

AI can act as a second decision-maker.

Human-AI conflict is a significant, emerging risk.

The conflict requires new measurement and risk assessment methods.

Abstract

The collaboration between humans and artificial intelligence (AI) is a significant feature in this digital age. However, humans and AI may have observation, interpretation, and action conflicts when working synchronously. This phenomenon is often masked by faults and, unfortunately, overlooked. This paper systematically introduces the human-AI conflict concept, causes, measurement methods, and risk assessment. The results highlight that there is a potential second decision-maker besides the human, which is the AI; the human-AI conflict is a unique and emerging risk in digitalized process systems; and this is an interdisciplinary field that needs to be distinguished from traditional fault and failure analysis; the conflict risk is significant and cannot be ignored.