Knowledge-intensive Language Understanding for Explainable AI

TL;DR

This paper emphasizes the importance of integrating explicit domain knowledge into explainable AI systems to enhance trust and human understanding of AI decision-making processes.

Contribution

It highlights the need for human-centered explanations that incorporate domain knowledge, advancing the development of trustworthy and transparent AI systems.

Findings

Explicit domain knowledge improves AI explainability.

Human-centered explanations foster greater trust in AI.

Involving domain expertise aligns AI decisions with human reasoning.

Abstract

AI systems have seen significant adoption in various domains. At the same time, further adoption in some domains is hindered by inability to fully trust an AI system that it will not harm a human. Besides the concerns for fairness, privacy, transparency, and explainability are key to developing trusts in AI systems. As stated in describing trustworthy AI "Trust comes through understanding. How AI-led decisions are made and what determining factors were included are crucial to understand." The subarea of explaining AI systems has come to be known as XAI. Multiple aspects of an AI system can be explained; these include biases that the data might have, lack of data points in a particular region of the example space, fairness of gathering the data, feature importances, etc. However, besides these, it is critical to have human-centered explanations that are directly related to decision-making similar to how a domain expert makes decisions based on "domain knowledge," that also include well-established, peer-validated explicit guidelines. To understand and validate an AI system's outcomes (such as classification, recommendations, predictions), that lead to developing trust in the AI system, it is necessary to involve explicit domain knowledge that humans understand and use.