Context-Aware Transformers For Spinal Cancer Detection and Radiological Grading

TL;DR

This paper introduces a novel transformer-based model, SCT, for analyzing vertebral structures in MR images, improving detection of spinal metastases, fractures, and degenerative disc changes by leveraging contextual information across the spinal column.

Contribution

We propose the Spinal Context Transformer (SCT), a new deep-learning architecture that considers all vertebral bodies and imaging modalities together for improved medical image analysis.

Findings

Strong agreement with radiologist annotations in metastases detection

Improved accuracy in degenerative disc grading

Effective use of multi-modal, multi-vertebra context

Abstract

This paper proposes a novel transformer-based model architecture for medical imaging problems involving analysis of vertebrae. It considers two applications of such models in MR images: (a) detection of spinal metastases and the related conditions of vertebral fractures and metastatic cord compression, (b) radiological grading of common degenerative changes in intervertebral discs. Our contributions are as follows: (i) We propose a Spinal Context Transformer (SCT), a deep-learning architecture suited for the analysis of repeated anatomical structures in medical imaging such as vertebral bodies (VBs). Unlike previous related methods, SCT considers all VBs as viewed in all available image modalities together, making predictions for each based on context from the rest of the spinal column and all available imaging modalities. (ii) We apply the architecture to a novel and important task: detecting spinal metastases and the related conditions of cord compression and vertebral fractures/collapse from multi-series spinal MR scans. This is done using annotations extracted from free-text radiological reports as opposed to bespoke annotation. However, the resulting model shows strong agreement with vertebral-level bespoke radiologist annotations on the test set. (iii) We also apply SCT to an existing problem: radiological grading of inter-vertebral discs (IVDs) in lumbar MR scans for common degenerative changes.We show that by considering the context of vertebral bodies in the image, SCT improves the accuracy for several gradings compared to previously published model.