Process Knowledge-Infused AI: Towards User-level Explainability, Interpretability, and Safety

TL;DR

This paper advocates for integrating process knowledge into AI systems to enhance explainability, interpretability, and safety, especially in high-stakes domains like healthcare and nutrition.

Contribution

It introduces a framework for infusing process knowledge into AI, enabling adherence to expert guidelines and providing human-understandable explanations.

Findings

Enhanced trust through user-understandable explanations

Improved safety by following expert-defined processes

Better compliance with domain-specific guidelines

Abstract

AI systems have been widely adopted across various domains in the real world. However, in high-value, sensitive, or safety-critical applications such as self-management for personalized health or food recommendation with a specific purpose (e.g., allergy-aware recipe recommendations), their adoption is unlikely. Firstly, the AI system needs to follow guidelines or well-defined processes set by experts; the data alone will not be adequate. For example, to diagnose the severity of depression, mental healthcare providers use Patient Health Questionnaire (PHQ-9). So if an AI system were to be used for diagnosis, the medical guideline implied by the PHQ-9 needs to be used. Likewise, a nutritionist's knowledge and steps would need to be used for an AI system that guides a diabetic patient in developing a food plan. Second, the BlackBox nature typical of many current AI systems will not work; the user of an AI system will need to be able to give user-understandable explanations, explanations constructed using concepts that humans can understand and are familiar with. This is the key to eliciting confidence and trust in the AI system. For such applications, in addition to data and domain knowledge, the AI systems need to have access to and use the Process Knowledge, an ordered set of steps that the AI system needs to use or adhere to.