Causality matters in medical imaging

TL;DR

This paper emphasizes the importance of causality in medical imaging to address data scarcity and mismatch issues, influencing data collection, annotation, and learning strategies for safer, more reliable machine learning models.

Contribution

It introduces a causal perspective to medical imaging challenges, offering new insights into data annotation, model generalization, and learning strategy selection, with practical recommendations.

Findings

Causal relationships impact model performance and generalization.

Semi-supervision may be unsuitable for image segmentation.

Causality informs better data collection and annotation strategies.

Abstract

This article discusses how the language of causality can shed new light on the major challenges in machine learning for medical imaging: 1) data scarcity, which is the limited availability of high-quality annotations, and 2) data mismatch, whereby a trained algorithm may fail to generalize in clinical practice. Looking at these challenges through the lens of causality allows decisions about data collection, annotation procedures, and learning strategies to be made (and scrutinized) more transparently. We discuss how causal relationships between images and annotations can not only have profound effects on the performance of predictive models, but may even dictate which learning strategies should be considered in the first place. For example, we conclude that semi-supervision may be unsuitable for image segmentation---one of the possibly surprising insights from our causal analysis, which is illustrated with representative real-world examples of computer-aided diagnosis (skin lesion classification in dermatology) and radiotherapy (automated contouring of tumours). We highlight that being aware of and accounting for the causal relationships in medical imaging data is important for the safe development of machine learning and essential for regulation and responsible reporting. To facilitate this we provide step-by-step recommendations for future studies.