Unsupervised Patch-GAN with Targeted Patch Ranking for Fine-Grained Novelty Detection in Medical Imaging

TL;DR

This paper introduces an unsupervised Patch-GAN framework that detects and localizes subtle anomalies in medical images by analyzing patches and ranking their abnormality scores, improving sensitivity to small deviations.

Contribution

The novel framework combines patch-based analysis with a ranking mechanism to enhance fine-grained anomaly detection in medical imaging without requiring labeled abnormal data.

Findings

Achieved AUCs of 95.79% on ISIC 2016 and 96.05% on BraTS 2019 datasets.

Outperformed three state-of-the-art baseline methods.

Effectively detects small and subtle anomalies in medical images.

Abstract

Detecting novel anomalies in medical imaging is challenging due to the limited availability of labeled data for rare abnormalities, which often display high variability and subtlety. This challenge is further compounded when small abnormal regions are embedded within larger normal areas, as whole-image predictions frequently overlook these subtle deviations. To address these issues, we propose an unsupervised Patch-GAN framework designed to detect and localize anomalies by capturing both local detail and global structure. Our framework first reconstructs masked images to learn fine-grained, normal-specific features, allowing for enhanced sensitivity to minor deviations from normality. By dividing these reconstructed images into patches and assessing the authenticity of each patch, our approach identifies anomalies at a more granular level, overcoming the limitations of whole-image evaluation. Additionally, a patch-ranking mechanism prioritizes regions with higher abnormal scores, reinforcing the alignment between local patch discrepancies and the global image context. Experimental results on the ISIC 2016 skin lesion and BraTS 2019 brain tumor datasets validate our framework's effectiveness, achieving AUCs of 95.79% and 96.05%, respectively, and outperforming three state-of-the-art baselines.