Large Scale Indexing of Generic Medical Image Data using Unbiased Shallow Keypoints and Deep CNN Features

TL;DR

This paper introduces a unified model combining shallow and deep features for large-scale medical image indexing, achieving high accuracy and efficiency in neuroimage classification tasks.

Contribution

It presents a novel Bayesian framework that integrates shallow and deep features, along with a domain adaptation strategy, for scalable neuroimage indexing.

Findings

State-of-the-art accuracy in neuroimage indexing

Efficient GPU-based implementation

Effective family and sex classification in large datasets

Abstract

We propose a unified appearance model accounting for traditional shallow (i.e. 3D SIFT keypoints) and deep (i.e. CNN output layers) image feature representations, encoding respectively specific, localized neuroanatomical patterns and rich global information into a single indexing and classification framework. A novel Bayesian model combines shallow and deep features based on an assumption of conditional independence and validated by experiments indexing specific family members and general group categories in 3D MRI neuroimage data of 1010 subjects from the Human Connectome Project, including twins and non-twin siblings. A novel domain adaptation strategy is presented, transforming deep CNN vectors elements into binary class-informative descriptors. A GPU-based implementation of all processing is provided. State-of-the-art performance is achieved in large-scale neuroimage indexing, both in terms of computational complexity, accuracy in identifying family members and sex classification.