Towards interpretable AI with quantum annealing feature selection

TL;DR

This paper introduces a quantum annealing-based method for interpreting CNNs in image classification, improving explanation quality and providing theoretical insights into the quantum optimization process.

Contribution

It proposes encoding feature importance selection as a quantum constrained optimization problem solved via quantum annealing, enhancing interpretability of CNNs.

Findings

Improved class disentanglement over GradCAM and GradCAM++

Enhanced explanation quality and transparency

Theoretical analysis of quantum annealing behavior

Abstract

Deep learning models are used in critical applications, in which mistakes can have serious consequences. Therefore, it is crucial to understand how and why models generate predictions. This understanding provides useful information to check whether the model is learning the right patterns, detect biases in the data, improve model design, and build systems that can be trusted. This work proposes a new method for interpreting Convolutional Neural Networks in image classification tasks. The approach works by selecting the most important feature maps that contribute to each prediction. To solve this combinatorial problem, we encode it into a quantum constrained optimization problem and propose to solve it using quantum annealing. We evaluate our method against the state-of-the-art explainable AI techniques, specifically GradCAM and GradCAM++, and observe an improved class disentanglement, i.e. the model's decision boundaries become more distinct and its reasoning more transparent. This demonstrates that our approach enhances the quality of explanations, making it easier to understand which features the model relies on for specific predictions. In addition, we study the computational behavior of the quantum annealing algorithm. Specifically, we analyze the minimum energy gap of the system during computation and the probability that the algorithm finds the correct solution. These analyses provide theoretical insight into why the method works effectively in practice.