Segment Using Just One Example

TL;DR

This paper introduces a novel single-example semantic segmentation method leveraging the Segment Anything model, which automatically generates prompts from one example image to segment target classes in query images without training.

Contribution

It presents a training-free, prompt-based segmentation approach that works with only one example image, eliminating the need for annotated datasets or text prompts.

Findings

Effective segmentation of building and car classes from a single example.

No training or text prompts required for the segmentation process.

Applicable to Earth observation images with promising results.

Abstract

Semantic segmentation is an important topic in computer vision with many relevant application in Earth observation. While supervised methods exist, the constraints of limited annotated data has encouraged development of unsupervised approaches. However, existing unsupervised methods resemble clustering and cannot be directly mapped to explicit target classes. In this paper, we deal with single shot semantic segmentation, where one example for the target class is provided, which is used to segment the target class from query/test images. Our approach exploits recently popular Segment Anything (SAM), a promptable foundation model. We specifically design several techniques to automatically generate prompts from the only example/key image in such a way that the segmentation is successfully achieved on a stitch or concatenation of the example/key and query/test images. Proposed technique does not involve any training phase and just requires one example image to grasp the concept. Furthermore, no text-based prompt is required for the proposed method. We evaluated the proposed techniques on building and car classes.