Combining Image Features and Patient Metadata to Enhance Transfer Learning

TL;DR

This study demonstrates that combining patient metadata with image features improves the classification performance of deep neural networks in medical imaging, with minimal additional computational cost.

Contribution

The paper introduces a method of integrating patient metadata with image features in transfer learning, showing consistent performance improvements across multiple neural network architectures.

Findings

Performance improved in all networks except VGG16

Metadata integration enhances classification accuracy

Minimal additional computational cost involved

Abstract

In this work, we compare the performance of six state-of-the-art deep neural networks in classification tasks when using only image features, to when these are combined with patient metadata. We utilise transfer learning from networks pretrained on ImageNet to extract image features from the ISIC HAM10000 dataset prior to classification. Using several classification performance metrics, we evaluate the effects of including metadata with the image features. Furthermore, we repeat our experiments with data augmentation. Our results show an overall enhancement in performance of each network as assessed by all metrics, only noting degradation in a vgg16 architecture. Our results indicate that this performance enhancement may be a general property of deep networks and should be explored in other areas. Moreover, these improvements come at a negligible additional cost in computation time, and therefore are a practical method for other applications.