Topological Data Analysis Guided Segment Anything Model Prompt Optimization for Zero-Shot Segmentation in Biological Imaging

TL;DR

This paper introduces a topological data analysis guided prompt optimization method for the Segment Anything Model, improving zero-shot biological image segmentation by enhancing small object detection and reducing computational complexity.

Contribution

It presents a novel TDA-based prompt optimization approach that replaces grid search, leveraging topological significance to improve segmentation performance.

Findings

Enhanced detection of small objects in biological images.

Significant reduction in computational complexity.

Preliminary results show improved segmentation quality.

Abstract

Emerging foundation models in machine learning are models trained on vast amounts of data that have been shown to generalize well to new tasks. Often these models can be prompted with multi-modal inputs that range from natural language descriptions over images to point clouds. In this paper, we propose topological data analysis (TDA) guided prompt optimization for the Segment Anything Model (SAM) and show preliminary results in the biological image segmentation domain. Our approach replaces the standard grid search approach that is used in the original implementation and finds point locations based on their topological significance. Our results show that the TDA optimized point cloud is much better suited for finding small objects and massively reduces computational complexity despite the extra step in scenarios which require many segmentations.