Extending 2D foundational DINOv3 representations to 3D segmentation of neonatal brain MR images

TL;DR

This paper introduces a novel 3D segmentation method that extends 2D foundation models to volumetric brain MRI analysis, enabling accurate hippocampal delineation in neonatal images.

Contribution

It proposes a structured window-based disassembly-reassembly mechanism to adapt 2D foundation features for 3D brain segmentation tasks.

Findings

Achieved a Dice score of 0.65 on hippocampal segmentation.

Demonstrated volumetric structure recovery from 2D foundation representations.

Provided a generalizable approach for 3D medical imaging applications.

Abstract

Precise volumetric delineation of hippocampal structures is essential for quantifying neurodevelopmental trajectories in pre-term and term infants, where subtle morphological variations may carry prognostic significance. While foundation encoders trained on large-scale visual data offer discriminative representations, their 2D formulation is a limitation with respect to the $3$D organization of brain anatomy. We propose a volumetric segmentation strategy that reconciles this tension through a structured window-based disassembly-reassembly mechanism: the global MRI volume is decomposed into non-overlapping 3D windows or sub-cubes, each processed via a separate decoding arm built upon frozen high-fidelity features, and subsequently reassembled prior to a ground-truth correspendence using a dense-prediction head. This architecture preserves constant a decoder memory footprint while forcing predictions to lie within an anatomically consistent geometry. Evaluated on the ALBERT dataset for hippocampal segmentation, the proposed approach achieves a Dice score of 0.65 for a single 3D window. The method demonstrates that volumetric anatomical structure could be recovered from frozen 2D foundation representations through structured compositional decoding, and offers a principled and generalizable extension for foundation models for 3D medical applications.