SegDINO: An Efficient Design for Medical and Natural Image Segmentation with DINO-V3

TL;DR

SegDINO introduces a lightweight, efficient segmentation framework that leverages a frozen DINOv3 backbone and a simple MLP head, achieving state-of-the-art results across medical and natural image datasets with minimal parameters.

Contribution

It presents a novel, parameter-efficient segmentation method combining a frozen DINOv3 encoder with a lightweight decoder, reducing complexity while maintaining high performance.

Findings

Achieves state-of-the-art results on six benchmarks.

Reduces model complexity and computational cost.

Effective across both medical and natural images.

Abstract

The DINO family of self-supervised vision models has shown remarkable transferability, yet effectively adapting their representations for segmentation remains challenging. Existing approaches often rely on heavy decoders with multi-scale fusion or complex upsampling, which introduce substantial parameter overhead and computational cost. In this work, we propose SegDINO, an efficient segmentation framework that couples a frozen DINOv3 backbone with a lightweight decoder. SegDINO extracts multi-level features from the pretrained encoder, aligns them to a common resolution and channel width, and utilizes a lightweight MLP head to directly predict segmentation masks. This design minimizes trainable parameters while preserving the representational power of foundation features. Extensive experiments across six benchmarks, including three medical datasets (TN3K, Kvasir-SEG, ISIC) and three natural image datasets (MSD, VMD-D, ViSha), demonstrate that SegDINO consistently achieves state-of-the-art performance compared to existing methods. Code is available at https://github.com/script-Yang/SegDINO.