More From Less: Self-Supervised Knowledge Distillation for Routine Histopathology Data

TL;DR

This paper introduces a self-supervised knowledge distillation method that leverages high-quality, information-dense histopathology data to improve classification accuracy on routine, information-sparse data, enhancing diagnostic insights without additional data collection.

Contribution

The study presents a novel self-supervised learning approach that distills knowledge from dense data into models usable with sparse data, improving accuracy and feature detection.

Findings

Enhanced classification accuracy on routine data

Identification of subtle, previously undetected features

Models incorporate insights from high-quality data

Abstract

Medical imaging technologies are generating increasingly large amounts of high-quality, information-dense data. Despite the progress, practical use of advanced imaging technologies for research and diagnosis remains limited by cost and availability, so information-sparse data such as H&E stains are relied on in practice. The study of diseased tissue requires methods which can leverage these information-dense data to extract more value from routine, information-sparse data. Using self-supervised deep learning, we demonstrate that it is possible to distil knowledge during training from information-dense data into models which only require information-sparse data for inference. This improves downstream classification accuracy on information-sparse data, making it comparable with the fully-supervised baseline. We find substantial effects on the learned representations, and this training process identifies subtle features which otherwise go undetected. This approach enables the design of models which require only routine images, but contain insights from state-of-the-art data, allowing better use of the available resources.