Bridging the gap between Natural and Medical Images through Deep Colorization

TL;DR

This paper introduces a novel deep learning approach that focuses on color adaptation to improve medical image recognition, especially in scenarios with limited data, by combining training from scratch with transfer learning.

Contribution

The work presents a dedicated color adaptation module integrated with transfer learning, enhancing diagnostic accuracy on X-ray images with scarce data.

Findings

Effective in data-scarce scenarios

Improves transfer of color information across datasets

End-to-end trainable architecture

Abstract

Deep learning has thrived by training on large-scale datasets. However, in many applications, as for medical image diagnosis, getting massive amount of data is still prohibitive due to privacy, lack of acquisition homogeneity and annotation cost. In this scenario, transfer learning from natural image collections is a standard practice that attempts to tackle shape, texture and color discrepancies all at once through pretrained model fine-tuning. In this work, we propose to disentangle those challenges and design a dedicated network module that focuses on color adaptation. We combine learning from scratch of the color module with transfer learning of different classification backbones, obtaining an end-to-end, easy-to-train architecture for diagnostic image recognition on X-ray images. Extensive experiments showed how our approach is particularly efficient in case of data scarcity and provides a new path for further transferring the learned color information across multiple medical datasets.