A generalizable 3D framework and model for self-supervised learning in medical imaging

TL;DR

This paper introduces 3DINO, a self-supervised learning framework for 3D medical imaging, enabling a general-purpose model that outperforms existing methods across diverse modalities and organs.

Contribution

The paper presents 3DINO, a novel SSL method for 3D medical images, and trains 3DINO-ViT on a large, multimodal dataset, demonstrating broad generalizability and superior performance.

Findings

3DINO-ViT outperforms state-of-the-art methods on multiple tasks

The framework generalizes across modalities and organs

The model performs well on out-of-distribution datasets

Abstract

Current self-supervised learning methods for 3D medical imaging rely on simple pretext formulations and organ- or modality-specific datasets, limiting their generalizability and scalability. We present 3DINO, a cutting-edge SSL method adapted to 3D datasets, and use it to pretrain 3DINO-ViT: a general-purpose medical imaging model, on an exceptionally large, multimodal, and multi-organ dataset of ~100,000 3D medical imaging scans from over 10 organs. We validate 3DINO-ViT using extensive experiments on numerous medical imaging segmentation and classification tasks. Our results demonstrate that 3DINO-ViT generalizes across modalities and organs, including out-of-distribution tasks and datasets, outperforming state-of-the-art methods on the majority of evaluation metrics and labeled dataset sizes. Our 3DINO framework and 3DINO-ViT will be made available to enable research on 3D foundation models or further finetuning for a wide range of medical imaging applications.