Weakly Supervised Pixel-Level Annotation with Visual Interpretability

TL;DR

This paper introduces an automated, explainable pixel-level annotation system for medical images that combines ensemble deep learning models, visual explanations, and uncertainty measures to improve accuracy and interpretability.

Contribution

It presents a novel ensemble approach integrating multiple pre-trained models with explainability and uncertainty quantification for weakly supervised pixel annotation.

Findings

Achieved 93.04% accuracy on TBX11K dataset.

Attained 96.4% accuracy on Fire segmentation dataset.

Produced pixel-level annotations with IoU scores of 36.07% and 64.7%.

Abstract

Medical image annotation is essential for diagnosing diseases, yet manual annotation is time-consuming, costly, and prone to variability among experts. To address these challenges, we propose an automated explainable annotation system that integrates ensemble learning, visual explainability, and uncertainty quantification. Our approach combines three pre-trained deep learning models - ResNet50, EfficientNet, and DenseNet - enhanced with XGrad-CAM for visual explanations and Monte Carlo Dropout for uncertainty quantification. This ensemble mimics the consensus of multiple radiologists by intersecting saliency maps from models that agree on the diagnosis while uncertain predictions are flagged for human review. We evaluated our system using the TBX11K medical imaging dataset and a Fire segmentation dataset, demonstrating its robustness across different domains. Experimental results show that our method outperforms baseline models, achieving 93.04% accuracy on TBX11K and 96.4% accuracy on the Fire dataset. Moreover, our model produces precise pixel-level annotations despite being trained with only image-level labels, achieving Intersection over Union IoU scores of 36.07% and 64.7%, respectively. By enhancing the accuracy and interpretability of image annotations, our approach offers a reliable and transparent solution for medical diagnostics and other image analysis tasks.