OBSeg: Accurate and Fast Instance Segmentation Framework Using Segmentation Foundation Models with Oriented Bounding Box Prompts

TL;DR

OBSeg is a novel instance segmentation framework for remote sensing images that leverages oriented bounding box prompts and segmentation foundation models to achieve high accuracy and speed, reducing reliance on bounding box detection performance.

Contribution

The paper introduces OBSeg, a new framework that uses OBB prompts with foundation models and a novel OBB prompt encoder, improving accuracy and efficiency in instance segmentation.

Findings

Outperforms existing methods in accuracy on multiple datasets.

Achieves competitive inference speed.

Effectively handles OBB prompts with a new encoder and distillation techniques.

Abstract

Instance segmentation in remote sensing images is a long-standing challenge. Since horizontal bounding boxes introduce many interference objects, oriented bounding boxes (OBBs) are usually used for instance identification. However, based on ``segmentation within bounding box'' paradigm, current instance segmentation methods using OBBs are overly dependent on bounding box detection performance. To tackle this problem, this paper proposes OBSeg, an accurate and fast instance segmentation framework using OBBs. OBSeg is based on box prompt-based segmentation foundation models (BSMs), e.g., Segment Anything Model. Specifically, OBSeg first detects OBBs to distinguish instances and provide coarse localization information. Then, it predicts OBB prompt-related masks for fine segmentation. Since OBBs only serve as prompts, OBSeg alleviates the over-dependence on bounding box detection performance of current instance segmentation methods using OBBs. Thanks to OBB prompts, OBSeg outperforms other current BSM-based methods using HBBs. In addition, to enable BSMs to handle OBB prompts, we propose a novel OBB prompt encoder. To make OBSeg more lightweight and further improve the performance of lightweight distilled BSMs, a Gaussian smoothing-based knowledge distillation method is introduced. Experiments demonstrate that OBSeg outperforms current instance segmentation methods on multiple datasets in terms of instance segmentation accuracy and has competitive inference speed. The code is available at https://github.com/zhen6618/OBBInstanceSegmentation.