Geometric Feature Prompting of Image Segmentation Models

TL;DR

This paper introduces a geometrically motivated prompt generator for image segmentation models like SAM, improving the automation and accuracy of segmenting complex scientific images such as plant roots with fewer prompts.

Contribution

It proposes a novel geometric prompt generation method that enhances segmentation performance in scientific images, specifically for plant root segmentation tasks.

Findings

Geometric prompts improve segmentation accuracy.

Fewer prompts needed for effective segmentation.

Open source software released for easy integration.

Abstract

Advances in machine learning, especially the introduction of transformer architectures and vision transformers, have led to the development of highly capable computer vision foundation models. The segment anything model (known colloquially as SAM and more recently SAM 2), is a highly capable foundation model for segmentation of natural images and has been further applied to medical and scientific image segmentation tasks. SAM relies on prompts -- points or regions of interest in an image -- to generate associated segmentations. In this manuscript we propose the use of a geometrically motivated prompt generator to produce prompt points that are colocated with particular features of interest. Focused prompting enables the automatic generation of sensitive and specific segmentations in a scientific image analysis task using SAM with relatively few point prompts. The image analysis task examined is the segmentation of plant roots in rhizotron or minirhizotron images, which has historically been a difficult task to automate. Hand annotation of rhizotron images is laborious and often subjective; SAM, initialized with GeomPrompt local ridge prompts has the potential to dramatically improve rhizotron image processing. The authors have concurrently released an open source software suite called geomprompt https://pypi.org/project/geomprompt/ that can produce point prompts in a format that enables direct integration with the segment-anything package.