SSplain: Sparse and Smooth Explainer for Retinopathy of Prematurity Classification

TL;DR

SSplain is a novel explainer method for ROP classification that produces realistic, smooth, and sparse pixel-wise explanations, improving interpretability and trust in neural network diagnoses.

Contribution

This paper introduces SSplain, a new explainer that enforces smoothness and sparsity in explanations, outperforming existing methods and aligning with clinical features.

Findings

SSplain achieves higher post-hoc accuracy and smoothness.

It identifies clinically relevant features consistent with domain knowledge.

SSplain generalizes well across multiple datasets.

Abstract

Neural networks are frequently used in medical diagnosis. However, due to their black-box nature, model explainers are used to help clinicians understand better and trust model outputs. This paper introduces an explainer method for classifying Retinopathy of Prematurity (ROP) from fundus images. Previous methods fail to generate explanations that preserve input image structures such as smoothness and sparsity. We introduce Sparse and Smooth Explainer (SSplain), a method that generates pixel-wise explanations while preserving image structures by enforcing smoothness and sparsity. This results in realistic explanations to enhance the understanding of the given black-box model. To achieve this goal, we define an optimization problem with combinatorial constraints and solve it using the Alternating Direction Method of Multipliers (ADMM). Experimental results show that SSplain outperforms commonly used explainers in terms of both post-hoc accuracy and smoothness analyses. Additionally, SSplain identifies features that are consistent with domain-understandable features that clinicians consider as discriminative factors for ROP. We also show SSplain's generalization by applying it to additional publicly available datasets. Code is available at https://github.com/neu-spiral/SSplain.