Colony Grounded SAM2: Zero-shot detection and segmentation of bacterial colonies using foundation models

TL;DR

This paper introduces Colony Grounded SAM2, a zero-shot detection and segmentation pipeline for bacterial colonies using foundation models, achieving high accuracy without additional training, and sharing resources openly for microbiology applications.

Contribution

We developed Colony Grounded SAM2, a novel zero-shot microbiological detection pipeline utilizing fine-tuned foundation models for robust bacterial colony analysis.

Findings

Achieved 93.1% mean Average Precision in detection.

Attained 0.85 Dice score for segmentation.

Demonstrated robustness on out-of-distribution datasets.

Abstract

The detection and classification of bacterial colonies in images of agar-plates is important in microbiology, but is hindered by the lack of labeled datasets. Therefore, we propose Colony Grounded SAM2, a zero-shot inference pipeline to detect and segment bacterial colonies in multiple settings without any further training. By utilizing the pre-trained foundation models Grounding DINO and Segment Anything Model 2, fine-tuned to the microbiological domain, we developed a model that is robust to data changes. Results showed a mean Average Precision of 93.1\% and a $Dice@detection$ score of 0.85, showing excellent detection and segmentation capabilities on out-of-distribution datasets. The entire pipeline with model weights are shared open access to aid with annotation- and classification purposes in microbiology.