Logic-Based Explainability in Machine Learning

TL;DR

This paper reviews research on formal, logic-based explanations for machine learning models, emphasizing the importance of rigorous, interpretable explanations for high-stakes applications.

Contribution

It provides an overview of the current state of formal explanation methods in ML, including definitions, complexity, logical encodings, and interpretability strategies.

Findings

Highlights the limitations of non-formal explanations

Summarizes logical approaches for rigorous explanations

Discusses challenges in making explanations human-interpretable

Abstract

The last decade witnessed an ever-increasing stream of successes in Machine Learning (ML). These successes offer clear evidence that ML is bound to become pervasive in a wide range of practical uses, including many that directly affect humans. Unfortunately, the operation of the most successful ML models is incomprehensible for human decision makers. As a result, the use of ML models, especially in high-risk and safety-critical settings is not without concern. In recent years, there have been efforts on devising approaches for explaining ML models. Most of these efforts have focused on so-called model-agnostic approaches. However, all model-agnostic and related approaches offer no guarantees of rigor, hence being referred to as non-formal. For example, such non-formal explanations can be consistent with different predictions, which renders them useless in practice. This paper overviews the ongoing research efforts on computing rigorous model-based explanations of ML models; these being referred to as formal explanations. These efforts encompass a variety of topics, that include the actual definitions of explanations, the characterization of the complexity of computing explanations, the currently best logical encodings for reasoning about different ML models, and also how to make explanations interpretable for human decision makers, among others.