Medical Image Retrieval using Deep Convolutional Neural Network

TL;DR

This paper presents a deep learning framework using CNNs for content-based medical image retrieval, achieving high classification accuracy and effective retrieval across multiple modalities and body organs.

Contribution

It introduces a CNN-based CBMIR system trained on a multi-modal dataset, effectively bridging the semantic gap in medical image retrieval.

Findings

Average classification accuracy of 99.77%

Mean average precision of 0.69 in retrieval

Effective retrieval across different body organs and modalities

Abstract

With a widespread use of digital imaging data in hospitals, the size of medical image repositories is increasing rapidly. This causes difficulty in managing and querying these large databases leading to the need of content based medical image retrieval (CBMIR) systems. A major challenge in CBMIR systems is the semantic gap that exists between the low level visual information captured by imaging devices and high level semantic information perceived by human. The efficacy of such systems is more crucial in terms of feature representations that can characterize the high-level information completely. In this paper, we propose a framework of deep learning for CBMIR system by using deep Convolutional Neural Network (CNN) that is trained for classification of medical images. An intermodal dataset that contains twenty four classes and five modalities is used to train the network. The learned features and the classification results are used to retrieve medical images. For retrieval, best results are achieved when class based predictions are used. An average classification accuracy of 99.77% and a mean average precision of 0.69 is achieved for retrieval task. The proposed method is best suited to retrieve multimodal medical images for different body organs.