Brain Tumor Image Retrieval via Multitask Learning

TL;DR

This paper introduces a multitask learning approach for brain tumor image retrieval that integrates multiple data aspects, resulting in richer representations than traditional single-task methods.

Contribution

It extends classification-based image retrieval with multitask learning to incorporate tumor type, shape, and localization information in CNN representations.

Findings

Multitask learning improves tumor representation quality.

The method outperforms single-task classification approaches.

Enhanced retrieval accuracy demonstrated on brain MRI data.

Abstract

Classification-based image retrieval systems are built by training convolutional neural networks (CNNs) on a relevant classification problem and using the distance in the resulting feature space as a similarity metric. However, in practical applications, it is often desirable to have representations which take into account several aspects of the data (e.g., brain tumor type and its localization). In our work, we extend the classification-based approach with multitask learning: we train a CNN on brain MRI scans with heterogeneous labels and implement a corresponding tumor image retrieval system. We validate our approach on brain tumor data which contains information about tumor types, shapes and localization. We show that our method allows us to build representations that contain more relevant information about tumors than single-task classification-based approaches.