GRASP-PsONet: Gradient-based Removal of Spurious Patterns for PsOriasis Severity Classification

TL;DR

This paper introduces a gradient-based interpretability framework to identify and remove problematic training images in psoriasis severity classification, enhancing model robustness and reducing annotation inconsistencies.

Contribution

The proposed method automatically flags spurious training images affecting model performance, improving generalization and reducing the need for manual annotation review.

Findings

Removing 8.2% of flagged images improves AUC-ROC by 5%

Detects over 90% of inter-rater disagreements by reviewing 30% of samples

Enhances robustness of remote psoriasis severity scoring models

Abstract

Psoriasis (PsO) severity scoring is important for clinical trials but is hindered by inter-rater variability and the burden of in person clinical evaluation. Remote imaging using patient captured mobile photos offers scalability but introduces challenges, such as variation in lighting, background, and device quality that are often imperceptible to humans but can impact model performance. These factors, along with inconsistencies in dermatologist annotations, reduce the reliability of automated severity scoring. We propose a framework to automatically flag problematic training images that introduce spurious correlations which degrade model generalization, using a gradient based interpretability approach. By tracing the gradients of misclassified validation images, we detect training samples where model errors align with inconsistently rated examples or are affected by subtle, nonclinical artifacts. We apply this method to a ConvNeXT based weakly supervised model designed to classify PsO severity from phone images. Removing 8.2% of flagged images improves model AUC-ROC by 5% (85% to 90%) on a held out test set. Commonly, multiple annotators and an adjudication process ensure annotation accuracy, which is expensive and time consuming. Our method detects training images with annotation inconsistencies, potentially removing the need for manual review. When applied to a subset of training data rated by two dermatologists, the method identifies over 90% of cases with inter-rater disagreement by reviewing only the top 30% of samples. This improves automated scoring for remote assessments, ensuring robustness despite data collection variability.