Exploiting Causality Signals in Medical Images: A Pilot Study with Empirical Results

TL;DR

This paper introduces a neural network-based method that leverages causal signals in medical images to improve classification accuracy and robustness, with promising results on prostate MRI and breast histopathology datasets.

Contribution

The study presents a novel causality-factors extractor module integrated into CNNs, enhancing interpretability and performance in medical image classification tasks.

Findings

Improved classification accuracy across datasets

Enhanced model robustness and focus on relevant image regions

Effective in both fully-supervised and few-shot learning

Abstract

We present a novel technique to discover and exploit weak causal signals directly from images via neural networks for classification purposes. This way, we model how the presence of a feature in one part of the image affects the appearance of another feature in a different part of the image. Our method consists of a convolutional neural network backbone and a causality-factors extractor module, which computes weights to enhance each feature map according to its causal influence in the scene. We develop different architecture variants and empirically evaluate all the models on two public datasets of prostate MRI images and breast histopathology slides for cancer diagnosis. We study the effectiveness of our module both in fully-supervised and few-shot learning, we assess its addition to existing attention-based solutions, we conduct ablation studies, and investigate the explainability of our models via class activation maps. Our findings show that our lightweight block extracts meaningful information and improves the overall classification, together with producing more robust predictions that focus on relevant parts of the image. That is crucial in medical imaging, where accurate and reliable classifications are essential for effective diagnosis and treatment planning.