Look Before You Leap! Designing a Human-Centered AI System for Change Risk Assessment

TL;DR

This paper presents a human-centered AI system for change risk assessment in production environments, integrating domain knowledge and continuous expert feedback to improve accuracy and address practical challenges.

Contribution

It introduces an end-to-end machine learning system that incorporates domain expertise and feedback loops for more effective risk assessment of change requests.

Findings

Addressed class imbalance and concept drift in the system

Implemented uncertainty estimation for predictions

Discussed scalability and practical deployment challenges

Abstract

Reducing the number of failures in a production system is one of the most challenging problems in technology driven industries, such as, the online retail industry. To address this challenge, change management has emerged as a promising sub-field in operations that manages and reviews the changes to be deployed in production in a systematic manner. However, it is practically impossible to manually review a large number of changes on a daily basis and assess the risk associated with them. This warrants the development of an automated system to assess the risk associated with a large number of changes. There are a few commercial solutions available to address this problem but those solutions lack the ability to incorporate domain knowledge and continuous feedback from domain experts into the risk assessment process. As part of this work, we aim to bridge the gap between model-driven risk assessment of change requests and the assessment of domain experts by building a continuous feedback loop into the risk assessment process. Here we present our work to build an end-to-end machine learning system along with the discussion of some of practical challenges we faced related to extreme skewness in class distribution, concept drift, estimation of the uncertainty associated with the model's prediction and the overall scalability of the system.