Visual Interpretability for Deep Learning: a Survey

TL;DR

This survey reviews recent advances in understanding and improving the interpretability of deep neural networks, especially CNNs, highlighting visualization, diagnosis, disentanglement, and future trends in explainable AI.

Contribution

It provides a comprehensive overview of methods for interpreting CNN representations and discusses future directions in explainable deep learning.

Findings

Visualization techniques help interpret CNN features

Disentangling representations improves model transparency

Future trends include explainable AI development

Abstract

This paper reviews recent studies in understanding neural-network representations and learning neural networks with interpretable/disentangled middle-layer representations. Although deep neural networks have exhibited superior performance in various tasks, the interpretability is always the Achilles' heel of deep neural networks. At present, deep neural networks obtain high discrimination power at the cost of low interpretability of their black-box representations. We believe that high model interpretability may help people to break several bottlenecks of deep learning, e.g., learning from very few annotations, learning via human-computer communications at the semantic level, and semantically debugging network representations. We focus on convolutional neural networks (CNNs), and we revisit the visualization of CNN representations, methods of diagnosing representations of pre-trained CNNs, approaches for disentangling pre-trained CNN representations, learning of CNNs with disentangled representations, and middle-to-end learning based on model interpretability. Finally, we discuss prospective trends in explainable artificial intelligence.