Concrete Safety for ML Problems: System Safety for ML Development and Assessment

TL;DR

This paper applies systems safety engineering principles to machine learning development and assessment to systematically address social and ethical risks, enhancing trustworthiness and safety in complex ML systems.

Contribution

It introduces a systematic safety framework for ML that considers interactions between development choices, environment, and societal impacts.

Findings

Demonstrates a safety approach for ML with social and ethical risks

Provides a systematic method for ML safety assurance

Enhances trustworthiness of ML systems in sociotechnical contexts

Abstract

Many stakeholders struggle to make reliances on ML-driven systems due to the risk of harm these systems may cause. Concerns of trustworthiness, unintended social harms, and unacceptable social and ethical violations undermine the promise of ML advancements. Moreover, such risks in complex ML-driven systems present a special challenge as they are often difficult to foresee, arising over periods of time, across populations, and at scale. These risks often arise not from poor ML development decisions or low performance directly but rather emerge through the interactions amongst ML development choices, the context of model use, environmental factors, and the effects of a model on its target. Systems safety engineering is an established discipline with a proven track record of identifying and managing risks even in high-complexity sociotechnical systems. In this work, we apply a state-of-the-art systems safety approach to concrete applications of ML with notable social and ethical risks to demonstrate a systematic means for meeting the assurance requirements needed to argue for safe and trustworthy ML in sociotechnical systems.