Exploring the Lottery Ticket Hypothesis with Explainability Methods: Insights into Sparse Network Performance

TL;DR

This paper investigates how pruning neural networks based on the Lottery Ticket Hypothesis affects their explainability and performance, revealing that pruning leads to less consistent concepts and pixels, which explains performance degradation.

Contribution

It introduces an explainability analysis of pruned networks using Grad-CAM and PCBMs, providing insights into the relationship between pruning, explainability, and performance.

Findings

Pruning causes performance to degrade gradually.

Discovered concepts and pixels become inconsistent after pruning.

Explainability tools reveal reasons for performance drops.

Abstract

Discovering a high-performing sparse network within a massive neural network is advantageous for deploying them on devices with limited storage, such as mobile phones. Additionally, model explainability is essential to fostering trust in AI. The Lottery Ticket Hypothesis (LTH) finds a network within a deep network with comparable or superior performance to the original model. However, limited study has been conducted on the success or failure of LTH in terms of explainability. In this work, we examine why the performance of the pruned networks gradually increases or decreases. Using Grad-CAM and Post-hoc concept bottleneck models (PCBMs), respectively, we investigate the explainability of pruned networks in terms of pixels and high-level concepts. We perform extensive experiments across vision and medical imaging datasets. As more weights are pruned, the performance of the network degrades. The discovered concepts and pixels from the pruned networks are inconsistent with the original network -- a possible reason for the drop in performance.