P2RBox: Point Prompt Oriented Object Detection with SAM

TL;DR

P2RBox introduces a novel point prompt-based method utilizing SAM for high-quality mask proposals and refined oriented object detection, significantly improving performance with minimal annotation effort.

Contribution

The paper presents P2RBox, a new approach that employs point prompts and SAM to generate accurate rotated boxes for oriented object detection, outperforming existing point-annotated methods.

Findings

Outperforms state-of-the-art point-annotated methods by 29% mAP.

Effectively integrates with multiple detectors to enhance detection accuracy.

Demonstrates practical application potential with minimal annotation effort.

Abstract

Single-point annotation in oriented object detection of remote sensing scenarios is gaining increasing attention due to its cost-effectiveness. However, due to the granularity ambiguity of points, there is a significant performance gap between previous methods and those with fully supervision. In this study, we introduce P2RBox, which employs point prompt to generate rotated box (RBox) annotation for oriented object detection. P2RBox employs the SAM model to generate high-quality mask proposals. These proposals are then refined using the semantic and spatial information from annotation points. The best masks are converted into oriented boxes based on the feature directions suggested by the model. P2RBox incorporates two advanced guidance cues: Boundary Sensitive Mask guidance, which leverages semantic information, and Centrality guidance, which utilizes spatial information to reduce granularity ambiguity. This combination enhances detection capabilities significantly. To demonstrate the effectiveness of this method, enhancements based on the baseline were observed by integrating three different detectors. Furthermore, compared to the state-of-the-art point-annotated generative method PointOBB, P2RBox outperforms by about 29% mAP (62.43% vs 33.31%) on DOTA-v1.0 dataset, which provides possibilities for the practical application of point annotations.