NeuroSeg Meets DINOv3: Transferring 2D Self-Supervised Visual Priors to 3D Neuron Segmentation via DINOv3 Initialization

TL;DR

This paper introduces a method to transfer 2D self-supervised visual features from DINOv3 to 3D neuron segmentation, improving accuracy and structural fidelity in volumetric neuroimaging with an inflation-based adaptation and topology-aware loss.

Contribution

We propose a novel inflation-based strategy to adapt 2D DINOv3 features to 3D neuron segmentation and introduce a topology-aware skeleton loss for better structural reconstruction.

Findings

Achieved an average of 2.9% improvement in Entire Structure Average

Demonstrated consistent accuracy gains across four neuronal datasets

Enhanced morphological fidelity in 3D neuronal reconstructions

Abstract

2D visual foundation models, such as DINOv3, a self-supervised model trained on large-scale natural images, have demonstrated strong zero-shot generalization, capturing both rich global context and fine-grained structural cues. However, an analogous 3D foundation model for downstream volumetric neuroimaging remains lacking, largely due to the challenges of 3D image acquisition and the scarcity of high-quality annotations. To address this gap, we propose to adapt the 2D visual representations learned by DINOv3 to a 3D biomedical segmentation model, enabling more data-efficient and morphologically faithful neuronal reconstruction. Specifically, we design an inflation-based adaptation strategy that inflates 2D filters into 3D operators, preserving semantic priors from DINOv3 while adapting to 3D neuronal volume patches. In addition, we introduce a topology-aware skeleton loss to explicitly enforce structural fidelity of graph-based neuronal arbor reconstruction. Extensive experiments on four neuronal imaging datasets, including two from BigNeuron and two public datasets, NeuroFly and CWMBS, demonstrate consistent improvements in reconstruction accuracy over SoTA methods, with average gains of 2.9% in Entire Structure Average, 2.8% in Different Structure Average, and 3.8% in Percentage of Different Structure. Code: https://github.com/yy0007/NeurINO.