SlimSAM: 0.1% Data Makes Segment Anything Slim

TL;DR

SlimSAM is a novel, data-efficient compression method for the Segment Anything Model that significantly reduces training data and model size while maintaining near-original performance through iterative pruning and distillation.

Contribution

Introduces SlimSAM, a new progressive pruning and distillation framework that achieves high compression with minimal training data for SAM.

Findings

Reduces training data requirement to 0.1% of original

Compresses model to 1.4% of parameters while maintaining performance

Achieves over 10x less training data than existing methods

Abstract

Current approaches for compressing the Segment Anything Model (SAM) yield commendable results, yet necessitate extensive data to train a new network from scratch. Employing conventional pruning techniques can remarkably reduce data requirements but would suffer from a degradation in performance. To address this challenging trade-off, we introduce SlimSAM, a novel data-efficient SAM compression method that achieves superior performance with extremely less training data. The essence of SlimSAM is encapsulated in the alternate slimming framework which effectively enhances knowledge inheritance under severely limited training data availability and exceptional pruning ratio. Diverging from prior techniques, our framework progressively compresses the model by alternately pruning and distilling distinct, decoupled sub-structures. Disturbed Taylor pruning is also proposed to address the misalignment between the pruning objective and training target, thereby boosting the post-distillation after pruning. SlimSAM yields significant performance improvements while demanding over 10 times less training data than any other existing compression methods. Even when compared to the original SAM, SlimSAM achieves approaching performance while reducing parameter counts to merely 1.4% (9.1M), MACs to 0.8% (23G), and requiring only 0.1% (10k) of the SAM training data. The code is available at http://github.com/czg1225/SlimSAM.