Collaborative Artificial Intelligence Needs Stronger Assurances Driven by Risks

TL;DR

This paper emphasizes the importance of stronger safety assurances in collaborative AI systems due to potential risks, and presents a multidisciplinary approach to develop risk-driven assurance processes.

Contribution

It introduces a novel risk-driven assurance framework for collaborative AI systems, addressing safety and compliance challenges in human-AI shared environments.

Findings

Identified emerging safety challenges in CAISs

Developed a multidisciplinary risk assurance process

Progress towards scalable safety standards for CAISs

Abstract

Collaborative AI systems (CAISs) aim at working together with humans in a shared space to achieve a common goal. This critical setting yields hazardous circumstances that could harm human beings. Thus, building such systems with strong assurances of compliance with requirements, domain-specific standards and regulations is of greatest importance. Only few scale impact has been reported so far for such systems since much work remains to manage possible risks. We identify emerging problems in this context and then we report our vision, as well as the progress of our multidisciplinary research team composed of software/systems, and mechatronics engineers to develop a risk-driven assurance process for CAISs.