MedCore: Boundary-Preserving Medical Core Pruning for MedSAM

TL;DR

MedCore is a structured pruning framework designed to compress medical segmentation models like MedSAM, effectively reducing parameters and FLOPs while maintaining high boundary and segmentation accuracy.

Contribution

It introduces a novel boundary-aware pruning method that preserves boundary structures and adapts to medical segmentation models, improving compression without sacrificing boundary fidelity.

Findings

MedCore reduces parameters by 60% and FLOPs by 58.4% with high segmentation accuracy.

It achieves 86.6% parameter reduction and 90.4G FLOPs with strong boundary quality.

MedSAM is in a head-fragile boundary regime, affecting pruning strategies.

Abstract

Medical segmentation foundation models such as SAM and MedSAM provide strong prompt-driven segmentation, but their image encoders are still too large for many clinical settings. Compression is also risky in medicine because a model can keep high Dice while losing boundary fidelity. We propose MedCore, a structured pruning framework for MedSAM. The main idea is to preserve two kinds of structures: structures that became important during SAM-to-MedSAM adaptation, and structures that have high boundary leverage. We identify the first type by a dual-intervention score that compares zeroing a group with resetting it to its original SAM weight. We identify the second type by boundary-aware Fisher estimation. We also introduce a boundary leverage principle, which shows that compression-induced boundary displacement is controlled by logit perturbation on the boundary divided by the logit spatial gradient. This principle explains why boundary metrics can degrade even when Dice remains high. On polyp segmentation benchmarks, MedCore reduces parameters by 60.0% and FLOPs by 58.4% while achieving Dice 0.9549, Boundary F1 0.6388, and HD95 5.14 after recovery fine-tuning. It also reaches 86.6% parameter reduction and 90.4G FLOPs with strong boundary quality. Our analysis further shows that MedSAM lies in a head-fragile boundary regime: head-pruning steps have 2.887 times larger 95th-percentile boundary leverage than MLP-pruning steps, and this logit-level effect is consistent with BF1 and HD95 degradation. Our code is available at https://github.com/cenweizhang/MedCore.