PlantSAM: An Object Detection-Driven Segmentation Pipeline for Herbarium Specimens

TL;DR

PlantSAM is an automated segmentation pipeline combining YOLOv10 and SAM2 that significantly improves herbarium image analysis by accurately isolating plant regions, leading to better classification performance.

Contribution

This paper introduces PlantSAM, a novel automated segmentation pipeline that integrates YOLOv10 and SAM2, specifically optimized for herbarium specimen images, achieving state-of-the-art segmentation accuracy.

Findings

Achieved IoU of 0.94 and Dice coefficient of 0.97 in segmentation.

Improved classification accuracy by up to 4.36%.

Enhanced F1-score by 4.15% through background removal.

Abstract

Deep learning-based classification of herbarium images is hampered by background heterogeneity, which introduces noise and artifacts that can potentially mislead models and reduce classification accuracy. Addressing these background-related challenges is critical to improving model performance. We introduce PlantSAM, an automated segmentation pipeline that integrates YOLOv10 for plant region detection and the Segment Anything Model (SAM2) for segmentation. YOLOv10 generates bounding box prompts to guide SAM2, enhancing segmentation accuracy. Both models were fine-tuned on herbarium images and evaluated using Intersection over Union (IoU) and Dice coefficient metrics. PlantSAM achieved state-of-the-art segmentation performance, with an IoU of 0.94 and a Dice coefficient of 0.97. Incorporating segmented images into classification models led to consistent performance improvements across five tested botanical traits, with accuracy gains of up to 4.36% and F1-score improvements of 4.15%. Our findings highlight the importance of background removal in herbarium image analysis, as it significantly enhances classification accuracy by allowing models to focus more effectively on the foreground plant structures.