Does Explainable Machine Learning Uncover the Black Box in Vision Applications?

TL;DR

This paper critically examines the effectiveness of explainable machine learning in vision applications, questioning whether current methods truly reveal the black box nature of models and proposing more rigorous principles for better explanations.

Contribution

It highlights limitations of current explainable ML approaches in vision and suggests incorporating more rigorous principles to improve understanding of black box models.

Findings

Current explainability methods may not effectively uncover model decision processes

Fundamental questions about explainability in vision ML remain inadequately addressed

Proposes perspectives for integrating rigorous principles into explainability approaches

Abstract

Machine learning (ML) in general and deep learning (DL) in particular has become an extremely popular tool in several vision applications (like object detection, super resolution, segmentation, object tracking etc.). Almost in parallel, the issue of explainability in ML (i.e. the ability to explain/elaborate the way a trained ML model arrived at its decision) in vision has also received fairly significant attention from various quarters. However, we argue that the current philosophy behind explainable ML suffers from certain limitations, and the resulting explanations may not meaningfully uncover black box ML models. To elaborate our assertion, we first raise a few fundamental questions which have not been adequately discussed in the corresponding literature. We also provide perspectives on how explainablity in ML can benefit by relying on more rigorous principles in the related areas.