DP2PNet: Diffusion-Based Point-to-Polygon Conversion for Single-Point Supervised Oriented Object Detection

TL;DR

This paper introduces DP2PNet, a new method for detecting oriented objects using single-point annotations, reducing the need for labor-intensive labeling.

Contribution

DP2PNet is the first diffusion model-based framework for single-point supervised oriented object detection.

Findings

DP2PNet achieves 53.82% and 53.61% mAP50 on DOTA-v1.0 and DIOR-R datasets, comparable to prior methods.

The model demonstrates strong noise robustness and cross-dataset generalization.

DP2PNet supports dynamic refinement stages without retraining, enabling flexible accuracy optimization.

Abstract

Rotated Bounding Boxes (RBBs) for oriented object detection are labor-intensive and time-consuming to annotate. Single-point supervision offers a cost-effective alternative but suffers from insufficient size and orientation information, leading existing methods to rely heavily on complex priors and fixed refinement stages. In this paper, we propose DP2PNet (Diffusion-Point-to-Polygon Network), the first diffusion model-based framework for single-point supervised oriented object detection. DP2PNet features three key innovations: (1) A multi-scale consistent noise generator that replaces manual or external model priors with Gaussian noise, reducing dependency on domain-specific information; (2) A Noise Cross-Constraint module based on multi-instance learning, which selects optimal noise point bags by fusing receptive field matching and object coverage; (3) A Semantic Key Point Aggregator that aggregates noise points via graph convolution to form semantic key points, from which pseudo-RBBs are generated using convex hulls. DP2PNet supports dynamic adjustment of refinement stages without retraining, enabling flexible accuracy optimization. Extensive experiments on DOTA-v1.0 and DIOR-R datasets demonstrate that DP2PNet achieves 53.82% and 53.61% mAP50, respectively, comparable to methods relying on complex priors. It also exhibits strong noise robustness and cross-dataset generalization.

Figures

Full text

1. Introduction

Oriented object detection has received widespread attention and research in fields containing complex scenes, such as retail scenes [1,2], scene text [3,4], and aerial images [5,6]. Compared to Horizontal Bounding Boxes (HBBs), Rotated Bounding Boxes (RBBs) can more accurately distinguish objects that are densely arranged or in various orientations [7].

Manual annotation of fine-grained RBBs is both time-consuming and labor-intensive. In contrast, prevalent coarse-grained annotation formats that reduce costs include image-level, HBB-level, and point-level annotations. Among these, image-level annotations are the most cost-effective; however, they offer limited performance in complex aerial images due to the absence of prior location information about objects. The use of HBB-level annotations [8,9] is primarily motivated by the goal of training rotated object detectors on datasets already equipped with such annotations. However, HBB-level annotations are still inefficient and labor-intensive, and they introduce additional noise, compounding the problem. Concluding that neither image-level nor HBB-level annotations are adequate for oriented object detection, we turn to the principles of weakly supervised learning to explore an annotation approach that optimizes performance while minimizing annotation effort.

Point-level annotation is widely utilized across various computer vision tasks [10,11,12,13,14], offering substantial cost reductions of 104.8% and 36.5% compared to RBB and HBB annotations [13], respectively. It is particularly effective for annotating densely packed or small objects in aerial images from a bird’s-eye view. In the realm of oriented object detection, only a limited number of researchers have explored point annotation [12,13,14], proposing a three-step pipeline to utilize it effectively: (1) Train a point label converter (PLC) using images annotated solely with points. (2) Use the trained PLC to convert point annotations into pseudo-RBB annotations. (3) Train any rotated object detector using these pseudo-labeled images in a supervised manner. The essence of this approach centers on the design of an accurate point label converter.

Due to the lack of two key prior information in point annotations: spatial size and orientation, existing PLCs primarily utilize a Multiple Instance Learning (MIL) approach. This strategy involves incorporating additional prior information to create proposal bags from point annotations, which are subsequently optimized based on category labels to select proposals with high classification confidence. Existing Point Label Converters (PLCs) are categorized based on the type of proposal: bounding box or pixel proposals. The former type utilizes manually designed hyperparameters such as size, aspect ratio, and angle to create various bounding box proposals. In contrast, the latter type depends on insights from large-scale models such as SAM [15] to generate pixel mask proposals. However, the two types of PLCs designed for predicting pseudo-rotated bounding boxes are highly dependent on the quality of proposals derived from prior information, which diminishes the algorithm’s generalization capability. Additionally, both PLC types use multiple cascaded stages to progressively refine the size and angle of the pseudo-rotated bounding boxes. Nonetheless, as is shown in Figure 1, the fixed number of refinement stages within the model constrains the algorithm’s performance. Therefore, we have identified the key issues that must be addressed in the design of an efficient and accurate PLC: First, how can we minimize reliance on complex prior information and proposal quality while still accurately generating pseudo-rotated bounding boxes? Second, how can we adjust the number of refinement stages flexibly to produce more precise pseudo-rotated bounding boxes without altering or retraining the model?

In this paper, we introduce the DiffusionPoint2Polygon network, which models Point Label Converters (PLC) as a task that generates inspirational sampling points within the feature map space. These points highlight the semantically significant parts of objects. Inspired by the denoising diffusion model process that gradually removes noise to generate images, we initially designed a multi-scale consistent noise generation module to facilitate the diffusion process. This module introduces multiple Gaussian noises into point annotations and maps them onto feature maps with varying receptive fields. Consequently, this results in the acquisition of several noise sampling point bags with differing receptive fields as proposals. This method of proposal generation, based on noise, reduces dependency on complex prior information. During the denoising process, the objective of this paper is not to entirely remove noise to recover the original point annotations but rather to generate pseudo-RBBs. Consequently, this paper introduces a Noise Cross-Constraint module that processes distinct bags of consistent noise points for each receptive field separately, employing Multiple Instance Learning (MIL) classifiers that match the number of receptive fields. This facilitates the initial identification of objects of various sizes and shapes. After initially identifying the noise sampling points within the object area, this paper designs a Semantic Key Point Aggregator that utilizes the graph convolutional network to consolidate these points into inspiration sampling points, representing semantically significant parts of the object. Subsequently, a pseudo-RBB can be generated for each object using the convex hull formed from these inspiration sampling points. This Point2Polygon method not only avoids direct prediction of the object’s orientation but also eliminates the need for precise segmentation of the object’s contour. Thanks to the progressive refinement characteristic of diffusion models, this paper demonstrates the flexibility to adjust the number of refinement stages by modifying the number of sampling steps during the denoising process, thereby achieving more precise pseudo-RBBs.

To further clarify the technical breakthroughs of this study, we compare this method with existing single-point supervised methods such as PointOBB and P2RBox in three dimensions: “prior type”, “optimization stage flexibility”, and “model framework”, as shown in Table 1. It can be seen that this method realizes the innovation of replacing manual prior with noise prior and fixed-stage optimization with dynamic progressive optimization, which effectively solves the key problems of existing methods.

Our main contributions are as follows:

• (1)This paper models oriented object detection under single-point supervision as the generation process of inspirational sampling points on the feature space. To our best knowledge, this is the first study to apply diffusion models to this field.
• (2)The DiffusionPoint2Polygon Network presented in this paper not only breaks away from the dependency on complex prior information through the multi-scale consistent noise point generation module during the diffusion process, but also incorporates a Noise Cross-Constraint module during the denoising process to perceive noise sampling points within a range of objects of various sizes and shapes. Subsequently, using the Semantic Key Point Aggregator, it generates pseudo-RBBs based on convex hulls formed by inspiration sampling points that indicate semantically important parts of the object.
• (3)Our method exhibits competitive performance on the DOTA and DIOR-R datasets relative to existing approaches that depend on intricate prior information, utilizing solely noise points as the basis for prior knowledge.

The rest of this paper is organized as follows: Section 2 reviews related work, including single-point supervised oriented object detection and the application of diffusion models in perception tasks. Section 3 details the architecture of DP2PNet, the design of core modules, and mathematical principles. Section 4 validates the effectiveness of the proposed method through dataset experiments, performance comparisons, and ablation studies. Section 5 and Section 6 summarizes the research findings and outlines future research directions.

2. Related Work

2.1. Fully Supervised Oriented Object Detection

Oriented object detection has increasingly garnered interest for its application in complex scenes, such as aerial images, where oriented bounding boxes offer enhanced compactness over traditional horizontal ones. Detectors in this domain are principally categorized into anchor-based and anchor-free types. Meanwhile, there are three main supervised training frameworks:

• (1)Image Supervision: Image-level supervision only annotates the categories of objects contained in an image, exemplified by the WSODet method.
• (2)HBB Supervision: As many existing datasets already have HBB annotations, another weakly supervised learning paradigm, represented by H2RBox and H2RBox-v2, utilizes HBBs as weak annotations to obtain RBBs. Their core is to use the self-supervised signal generated by actively rotating images and the inherent symmetry of objects to accomplish the transition from HBBs to RBBs.
• (3)Point Supervision: Point-level annotation provides stronger prior information regarding the object location compared to image-level annotation, while adding relatively minimal annotation costs. Additionally, the cost of point-level annotation is relatively low when compared to other instance-level annotations, such as horizontal or rotated boxes. In the field of oriented object detection, point-level supervision remains a relatively new innovation. Current research primarily focuses on designing a label converter that can transform point annotations into RBBs. PointOBB enhances the MIL network’s ability to perceive the scale and orientation of objects by scaling and rotating the original image, while P2RBOX introduces SAM as a mask generator and designs an inspector module to select high-quality masks for generating RBBs. Label converters can effectively be integrated with the now mature RBox-supervised methods. However, existing label converters heavily rely on the quality of prior information and multi-stage progressive refinement.

2.2. Diffusion Model for Perception Tasks

As a class of deep generative models, diffusion models [16,17], start from the sample in random sample distribution and recover the data sample via a gradual denoising process. Diffusion models have recently achieved significant success in the field of image generation [18] in computer vision, and DiffusionDet [19] has also initially explored its potential in the domain of image discriminative tasks.

(1)Application of Diffusion Models in General Object Detection: In recent years, diffusion models have been gradually applied to general object detection tasks. DiffusionDet [19] proposes to model the object detection process as a denoising diffusion process of bounding box coordinates, generating candidate boxes from random noise and gradually optimizing them. DiffusionRPN [20] applies the diffusion model to the region proposal network, generating high-quality candidate regions through noise denoising, which improves the recall rate of small targets. Wang [21] designs a dynamic diffusion optimization module to adjust the denoising steps according to the quality of candidate boxes, realizing adaptive optimization of detection results. These methods all follow the “noise generation—step-by-step denoising” framework, which provides a new idea for solving the problem of object detection.(2)Research Gaps of Diffusion Models in Oriented Detection: Although diffusion models have shown potential in general object detection, there are still three unsolved problems in oriented detection scenarios: (1) The existing methods do not adapt to the rotation angle prediction requirement of oriented detection. The general object detection only needs to predict the horizontal bounding box, while the oriented detection needs to further predict the rotation angle, and the diffusion model lacks the corresponding angle optimization mechanism. (2) The existing methods are not optimized for the scenario of less annotation information in single-point supervision. Most diffusion-based detection methods rely on full supervision information (such as accurate bounding box annotations) and cannot effectively use the limited information of single-point annotations. (3) The existing methods do not solve the problem of noise adaptation of multi-scale targets. The size difference of targets in oriented detection (especially aerial image detection) is large, and the fixed noise distribution of the existing methods cannot adapt to targets of different scales.

2.3. Positioning of This Study

This study fills the above research gaps from three aspects: (1) By designing a Semantic Key Point Aggregator, the inspiration point set is obtained through graph convolution aggregation of noise points, and the convex hull of the inspiration point set is used to generate pseudo-RBBs, which realizes the angle prediction in oriented detection without directly predicting the angle. (2) Using noise prior instead of manual prior or external model prior, the model can effectively learn from single-point annotation information, reducing the dependence on annotation quantity. (3) By designing a multi-scale consistent noise generator, the noise variance is adaptively adjusted according to the scale of the feature map, realizing the noise adaptation of multi-scale targets.

Table 2 systematically compares DP2PNet with three representative single-point supervised oriented object detection methods across five core dimensions. It highlights DP2PNet’s innovations: replacing manual/external model priors with Gaussian noise, enabling dynamic refinement stages (1–7 stages) instead of fixed ones, and integrating diffusion models with MIL and graph convolution. Notably, DP2PNet avoids direct angle prediction by leveraging convex hulls of semantic key points, achieving competitive mAP50 (53.82%) on DOTA while enhancing generalization and reducing prior dependence.

3. Method

3.1. Preliminaries

(1)Point label converter:

The fundamental concept of the point label converter is to transform individual point annotations, $[eqn]$ , within image $[eqn]$ into RBB annotations, $[eqn]$ . Each point annotation $[eqn]$ is represented as $[eqn]$ , where $[eqn]$ specifies the position, and c denotes the category. Similarly, each RBB annotation $[eqn]$ is defined as $[eqn]$ , where $[eqn]$ indicates the center position of the RBB, $[eqn]$ its width and height, and a the angle. Consequently, the point label converter is designed to address the absence of scale and orientation information in point annotations compared to RBBs.

(2)Diffusion Model:

Recent diffusion models usually use two Markov chains: a forward chain that perturbs the image to noise and a reverse chain that refines noise back to the image. Formally, given a data distribution $[eqn]$ , the forward noise perturbing process at time t is defined as $[eqn]$ . It gradually adds Gaussian noise to the data according to a variance schedule $[eqn]$ :

$$[eqn]$$

where $[eqn]$ is the conditional probability distribution of the noisy point set $[eqn]$ at time step t given $[eqn]$ (following a Gaussian distribution); $[eqn]$ denotes a Gaussian distribution with mean $[eqn]$ and covariance matrix $[eqn]$ ; $[eqn]$ and $[eqn]$ are the point sets at time steps t and $[eqn]$ in the diffusion process; $[eqn]$ is the variance schedule parameter controlling noise intensity; I is the identity matrix.

Given $[eqn]$ , we can easily obtain a sample of $[eqn]$ by sampling a Gaussian vector $[eqn]$ and applying the transformation as follows:

$$[eqn]$$

where $[eqn]$ is the original single-point annotation (clean data, $[eqn]$ with position $[eqn]$ and category c); $[eqn]$ simplifies forward noise perturbation; $[eqn]$ is Gaussian noise sampled from $[eqn]$ ; other symbols are consistent with Equation (1).

In this work, we aim to model the point label converter through diffusion models. In our setup, the data sample is a single-point annotation $[eqn]$ , and the point label converter $[eqn]$ acquires perception of object size and orientation in the process of predicting $[eqn]$ from the noise point $[eqn]$ given the corresponding image $[eqn]$ . The point label converter can then construct pseudo-RBBs for each single-point annotation based on noise points with the most appropriate sampling steps during the iterative process in the inference stage.

3.2. Architecture and Pipeline

(1)Architecture & Training:

The overall framework of the point label converter constructed 216 in this paper is shown in Figure 2. During the denoising process, this paper introduces the Noise Cross-Constraint (NCC) module and the Semantic Key Point Aggregator (SKPA)—both designed to reverse the diffusion process while enabling the model to discern object size and identify categorically valuable parts.

The NCC module focuses on perceiving objects of diverse sizes and shapes. It uses M sets of noise point bags with different receptive fields for the u-th object ( $[eqn]$ ) to match corresponding MIL classifiers, generating classification scores $[eqn]$ . Each MIL classifier includes two branches: a category-discriminating classification branch and a foreground-background discriminating instance branch. Summing the scores of all points in each noise point bag yields the bag’s total score $[eqn]$ , which is used to compute the multi-instance loss ( $[eqn]$ ). This design allows the module to detect objects across scales via the receptive fields of noise points.

Furthermore, the NCC module integrates a Noise Consistency Loss ( $[eqn]$ ) based on self-supervised learning. This loss minimizes score disparities among noise point bags of the same object sharing the same receptive field, enhancing the model’s ability to consistently recognize objects of varying shapes.

To align aggregated points with semantically significant parts of the object, the Semantic Key Point Aggregator (SKPA) aggregates noise sampling points from the selected noise point bag via graph convolution, producing semantic key points $[eqn]$ . A Multilayer Perceptron-based classification head calculates the point loss ( $[eqn]$ ) for these aggregated key points.

Additionally, the Jarvis March algorithm is used to compute the convex hull of the semantic key points, and RROI Align extracts region-specific features to calculate the classification loss ( $[eqn]$ ). This dual-loss design ensures the semantic key points align with the object’s critical classification regions while comprehensively covering the entire object.

(2)Inference:

Similar to other diffusion-based methods, the inference process of the point label converter involves a denoising procedure—transforming Gaussian-distributed noisy points into annotation-aligned points through multiple sampling steps. However, distinct from standard diffusion models, our weakly supervised point label converter aims to construct RBBs during denoising rather than fully recovering the original point annotations.

In the denoising process, DP2PNet first leverages the Noise Cross-Constraint module to select the optimal feature map $[eqn]$ that matches the object size, based on the classification scores of noise point bags $[eqn]$ . From the noise point bags corresponding to $[eqn]$ , it further selects the bag $[eqn]$ with the best object coverage.

Subsequently, DP2PNet evaluates two key criteria for the current-stage inspiration sampling point bag $[eqn]$ : whether it identifies the most classification-valuable parts of the object, and whether it fully encompasses the object. If denoising is insufficient (i.e., the criteria are not met), the model uses DDIM [17] to estimate the next-stage inspiration sampling point bag $[eqn]$ from $[eqn]$ .

To align inference with the training process (which incorporates noise injection), the model supplements $[eqn]$ with new random Gaussian noise points. This flexible adjustment of denoising steps avoids two critical issues: overly broad noise point coverage due to insufficient denoising and incomplete object coverage due to excessive denoising.

3.3. Multi-Scale Consistent Noise Generator

Due to point annotations lacking information on object scale and orientation compared to RBB annotations, the existing point label converter mainly relies on artificially designed priors composed of a variety of sizes, aspect ratios, and combinations of rotation angles, or introduces prior models such as SAM to provide information on object scale and orientation. To minimize the dependence of the Point Label Converters on prior information, this paper completes the diffusion process in the diffusion model paradigm through the following three steps:

• (1)Sampling Point Padding: Due to the diversity of object scales, to ensure an adequate number of sampling points covers the object, we first add some extra points to the point annotations $[eqn]$ for each object u. Consequently, each object u is associated with a noise point bag $[eqn]$ containing a fixed number of points N. We explored several padding strategies, such as duplicating existing point annotations or connecting random points. Among these, connecting random points yielded the best results.
• (2)Single-scale Consistent Noise Generation: We add Gaussian noise to the set of object-padded points $[eqn]$ at the u-th location, where the noise scale is controlled by $[eqn]$ , and $[eqn]$ employs a monotonically decreasing cosine schedule across different time steps t. Due to the limited receptive field of individual noise sampling points, a point bag containing a fixed number N of points may not effectively cover the object after adding noise generated by a single random seed. Adding more sampling points directly into the point set package $[eqn]$ would be difficult to optimize due to insufficient supervision information. Therefore, at time step t, this paper uses K different random seeds to generate K groups of different noise point bags $[eqn]$ for each point set package $[eqn]$ .
• (3)Multi-scale Noise Sampling Point Mapping: Given the diversity of object scales, the size of the parts of objects most valuable for classification also varies. Thus, the noise sampling point bag $[eqn]$ is mapped onto the multi-scale feature map $[eqn]$ , enabling each sampling point to obtain deep features with varying receptive fields. The specific process is formalized as follows:

$$[eqn]$$

where $[eqn]$ is the noise point bag mapped to the m-th multi-scale feature map; $[eqn]$ maps noise point coordinates to $[eqn]$ for deep feature extraction; $[eqn]$ is the original noise point bag set; $[eqn]$ is the m-th FPN output feature map ( $[eqn]$ for P2/P3/P4); M is the number of multi-scale feature layers.

3.4. Noise Cross-Constraint Module

To learn the perception ability of objects at different scales, this module first refers to WSDDN [22] and designs three corresponding dual-stream structures for noise point bags with different receptive fields as the MIL classifier. Specifically, for the K noise point bags $[eqn]$ on the i-th feature map with the number of channels D corresponding to the u-th object, this paper feeds it into the classification branch $[eqn]$ to obtain $[eqn]$ , then uses an activation function to obtain classification scores $[eqn]$ , where N is the number of points in the noise point bag, and C represents the number of categories. Similarly, the instance scores $[eqn]$ are also obtained through the instance branch $[eqn]$ and an activation function.

$$[eqn]$$ $$[eqn]$$

where $[eqn]$ is the raw output of the classification branch for the K noise point bags of the u-th object on the i-th feature map; $[eqn]$ is the classification branch network; $[eqn]$ are the mapped noise point bags ( $[eqn]$ ); $[eqn]$ are the classification scores ( $[eqn]$ ); $[eqn]$ is the SoftMax activation function; C is the number of object categories. $[eqn]$ is the raw output of the instance branch; $[eqn]$ is the instance branch network; $[eqn]$ are the foreground-background discrimination scores ( $[eqn]$ ); $[eqn]$ is the Sigmoid activation function; other symbols are consistent with Equation (4).

By calculating the Hadamard product of $[eqn]$ and $[eqn]$ , the noise point score $[eqn]$ can be obtained. The noise point bag score $[eqn]$ can be calculated by summing the scores of N noise points in each noise point bag.

$$[eqn]$$ $$[eqn]$$

where $[eqn]$ is the combined score of each noise point; ⊙ denotes the Hadamard product (element-wise multiplication); other symbols are consistent with Equations (4) and (5). $[eqn]$ is the total score of each noise point bag (sum of N point scores); $[eqn]$ sums over all points in a bag; other symbols are consistent with Equation (6).

The noise point bag score $[eqn]$ can be seen as a weighted sum of classification scores and instance scores. The MIL loss for noise point bags with the i-th receptive field uses the form of cross-entropy loss, defined as:

$$[eqn]$$

where $[eqn]$ is the MIL loss for the i-th feature map; R is the number of objects in the image; $[eqn]$ is the focal loss function; $[eqn]$ is the total score of the j-th bag of the u-th object; $[eqn]$ is the ground-truth category of the u-th object.

The MIL Loss of the Noise Cross-Constraint module is the sum of the MIL loss of all receptive field noise point bags, as shown in the following formula:

$$[eqn]$$

where $[eqn]$ is the total MIL loss (sum over M feature maps); $[eqn]$ is the per-feature-map MIL loss; M is the number of multi-scale feature layers.

In an ideal scenario, the classification scores of noise point bags with the same receptive field should remain consistent. Therefore, we use cosine similarity to measure the pairwise similarity between the classification scores of noise point bags packages in the i-th receptive field.

$$[eqn]$$

where $[eqn]$ is the cosine similarity between the j-th and t-th bag scores (range [0, 2]); $[eqn]$ is their dot product; $[eqn]$ and $[eqn]$ are their L2 norms.

After conducting similarity measurements, we can obtain the consistency loss of the i-th receptive field as well as the consistency loss of the Noise Cross-Constraint module.

$$[eqn]$$

where $[eqn]$ is the consistency loss for the i-th feature map; $[eqn]$ is the Smooth L1 loss with target 0; $[eqn]$ is the total consistency loss; other symbols are consistent with Equation (10).

Finally, the loss of the Noise Cross-Constraint module $[eqn]$ is the sum of the $[eqn]$ and the consistency loss.

$$[eqn]$$

where $[eqn]$ is the total loss of the Noise Cross-Constraint module; $[eqn]$ and $[eqn]$ are loss weights; other symbols are consistent with Equations (9) and (11).

Two core indicators guide the selection of optimal noise point bags (1) Receptive Field Matching (RFM): Measures the adaptability between the receptive field size of the feature map and the actual object size, defined as $[eqn]$ , where $[eqn]$ is the receptive field size of the feature map, and $[eqn]$ is the actual area of the ground-truth object; (2) Object Coverage (OC): The proportion of noise points in the bag that fall within the ground-truth object region, defined as $[eqn]$ , where $[eqn]$ is the number of noise points inside the object, and $[eqn]$ is the total number of points in the bag. The optimal noise point bag is selected based on the weighted fusion score $[eqn]$ , with weights determined via cross-validation to balance scale adaptability and region completeness.

3.5. Semantic Key Point Aggregator

As individual noise sampling points in the noise point bags have limited receptive fields and cannot provide overall information about the object, and to avoid direct estimation of the target direction or detailed depiction of the target outline, inspired by RepPoints [23], this paper designs an Semantic Key Point Aggregator (SKPA) to aggregate the sampling points in the noise point bags to obtain a bag of inspiration sampling points that indicate semantically important local areas of the object. However, since point annotations do not provide any size information, we cannot use attention mechanisms to offset the sampling points. Therefore, inspired by fine-grained image classification methods [24,25], we utilize graph convolution [26] to interact the information between each noise sampling point and aggregate them to obtain a bag of inspiration sampling points. Finally, this paper uses the Jarvis March algorithm [27,28] to obtain the convex hull of the bag of inspiration sampling points to obtain the corresponding pseudo-RBB.

The Semantic Key Point Aggregator encapsulates each noise sampling point in the noise point bag $[eqn]$ , where $[eqn]$ and $[eqn]$ , as a graph node. It employs graph convolutional networks to aggregate them based on the feature similarity between the noise sampling points. The pairwise adjacency relationship between different nodes can be defined as:

$$[eqn]$$

where $[eqn]$ is the adjacency weight between the m-th and n-th noise points; $[eqn]$ is a 1 × 1 convolution layer for feature projection; $[eqn]$ are noise points in $[eqn]$ ; other terms denote dot product and L2 norm of projected features.

Subsequently, through this densely connected GCN operation, each node can be updated via this similarity-based aggregation:

$$[eqn]$$

where $[eqn]$ is the learnable graph weights with the hidden dimension $[eqn]$ , and $[eqn]$ is the diagonal matrix for normalization. $[eqn]$ is the matrix form of the noise point bag. Thus, the feature of each node is updated by this message passing operation. Another purpose of the graph embedding operation is to divide the nodes in the graph into multiple groups. This paper learns the grouping rules by introducing a new graph convolution layer $[eqn]$ , which aggregates the noisy point bag $[eqn]$ .

$$[eqn]$$

where $[eqn]$ is the graph weights. Thus, $[eqn]$ defines a mapping function to aggregate the original N noise sampling points in the noise point bag into E inspiration sample points. This paper uses $[eqn]$ to map the coordinates and features of N noise sampling points to generate the coordinates and features of E inspiration sample points. In this paper, the SoftMax operation is used on the group dimension of $[eqn]$ , so the coordinates and features of each inspiration sample point can be viewed as a probabilistic weighted combination of the coordinates and features of the original N noise sample points. After obtaining the $[eqn]$ , this paper concatenates the features corresponding to the inspiration sample points in the bag $[eqn]$ , feeds it into the fully connected layer $[eqn]$ , applies the SoftMax activation function to obtain the corresponding classification scores $[eqn]$ , and calculates the loss with the annotated category $[eqn]$ , as shown in the following equation:

$$[eqn]$$ $$[eqn]$$

where $[eqn]$ are the classification scores of semantic key point bags ( $[eqn]$ ); $[eqn]$ is the fully connected layer; $[eqn]$ merges features of E key points; $[eqn]$ is the set of semantic key point bags. $[eqn]$ is the point loss encouraging key points to align with semantically important parts; other symbols are consistent with Equations (8) and (16).

Subsequently, this paper utilizes RROI Align [27] to extract features of the convex hull regions corresponding to the inspiration sample points bag, and feeds it into the fully connected layer $[eqn]$ . The SoftMax activation function is applied to obtain the corresponding classification scores $[eqn]$ , and the classification is calculated as shown in the following equation:

$$[eqn]$$ $$[eqn]$$

The loss of the Semantic Key Point Aggregator $[eqn]$ is the sum of $[eqn]$ and $[eqn]$ . The former encourages the module to find inspiration sample points that align with the most class-valuable parts of the object, while the latter motivates the module to ensure that the area surrounded by the insightful sample points found also has good classification ability to more comprehensively cover the object.

3.6. Theoretical Analysis

3.6.1. Mapping Relationship Between Diffusion Process and Pseudo-Box Generation

We define the probability distribution of the noise point set at time step t as $[eqn]$ , which follows a Gaussian distribution $[eqn]$ . As the number of diffusion steps T increases, when $[eqn]$ , $[eqn]$ , and $[eqn]$ , which is the initial random noise distribution. During the denoising process, the model learns to map the noise point set $[eqn]$ back to the original point annotation $[eqn]$ . However, in this paper, we do not need to completely recover $[eqn]$ , but to generate a set of inspiration points whose convex hull can approximate the real RBB.

We assume that the real RBB of the object is $[eqn]$ , and the convex hull of the inspiration point set is $[eqn]$ . We need to prove that the IoU between $[eqn]$ and B has a lower bound. According to the Jarvis March algorithm, the convex hull $[eqn]$ is the smallest convex polygon that contains all the inspiration points. For any inspiration point $[eqn]$ , it satisfies $[eqn]$ with high probability (due to the constraint of the Noise Cross-Constraint module and the Semantic Key Point Aggregator). Therefore, $[eqn]$ with high probability, and the area of $[eqn]$ is at least the area of the smallest convex polygon containing the semantically important parts of the object. Let the area of the semantically important parts be $[eqn]$ , then the area of $[eqn]$ is at least $[eqn]$ . The area of B is $[eqn]$ . Therefore, the IoU between $[eqn]$ and B is at least $[eqn]$ . In practice, $[eqn]$ accounts for a certain proportion of $[eqn]$ (usually more than 50% for most objects), so the IoU has a lower bound of about 0.5.

3.6.2. Convergence Analysis of Noise Cross-Constraint Module

We analyze the convergence of the loss function of the Noise Cross-Constraint module $[eqn]$ . We assume that the learning rate is $[eqn]$ , and the gradient of the loss function with respect to the model parameters $[eqn]$ is $[eqn]$ . According to the gradient descent method, the update rule of the parameters is $[eqn]$ .

We first analyze the Lipschitz constant of $[eqn]$ . The focal loss function $[eqn]$ is a convex function, and its derivative is bounded. The MIL loss $[eqn]$ is the average of the focal losses of multiple noise point bags, so it is also a convex function, and its Lipschitz constant $[eqn]$ is bounded. Similarly, the consistency loss $[eqn]$ is a smooth function, and its Lipschitz constant $[eqn]$ is also bounded. Therefore, the Lipschitz constant of $[eqn]$ is $[eqn]$ , which is bounded.

According to the convergence theory of gradient descent, when the learning rate $[eqn]$ , the loss function $[eqn]$ is monotonically decreasing, and the parameters $[eqn]$ converge to the optimal solution. In this paper, we set the learning rate to 0.001 through cross-validation, which satisfies $[eqn]$ , ensuring the convergence of the Noise Cross-Constraint module.

4. Experiment

4.1. Datasets and Implementation Details

(1)Datasets: DOTA-v1.0 [29] is presently among the most widely employed datasets for oriented object detection in aerial images. It comprises 2806 images, 188,282 instances annotated with RBoxes, and is classified into 15 categories. For training and testing, we follow a standard protocol by cropping images into 1024 × 1024 patches with a stride of 824. DIOR-R [30] is an aerial image dataset annotated by RBoxes based on its horizontal annotation version DIOR. The dataset consists of 23,463 images, 190,288 instances, and is classified into 20 categories.(2)Single-Point Annotation: In order to accurately simulate manually annotated point annotations, this paper does not directly use the center point of the RBB label as the point annotation. Instead, it selects random points within a range of 10% relative to the width and height of the RBB near the center point as the single-point annotations, thereby reproducing the deviations in manual annotations. The impact of the deviation range will be discussed in Section 5.(3)Training Details: The algorithms employed in the experiments of this paper are from the open-source library MMRotate [31] based on Pytorch. This paper follows the default settings in MMRotate. The experiments in this paper were conducted on the NVIDIA 4090 with 24GB of memory. For training the DP2PNet with single-point annotations and the fully supervised rotation box algorithm trained with pseudo-RBBs generated by the DP2PNet in this paper, a “1×” schedule including 12 epochs was used. For the compared algorithms, we followed their base settings.(4)Evaluation Metric: Mean Average Precision (mAP) is used as the main metric to compare our method in this paper with existing methods. To evaluate the quality of pseudo-RBBs generated by DP2PNet from point annotations, this paper reports the mean Intersection over Union (mIoU) between manually annotated RBBs (GT) and pseudo-RBBs.

DP2PNet adopts single-point supervision, and its annotation cost is 36.5% lower than HBox supervision and 104.8% lower than RBox supervision. Although its mAP50 is slightly lower than HBox-supervised methods (such as H2RBox-v2), the annotation cost is only 1/10 of the latter, achieving excellent cost-effectiveness.

4.2. Performance Comparisons

4.2.1. Comparisons Results

To verify the performance of DP2PNet in single-point supervised oriented detection, we compared it with existing methods on the DOTA-v1.0 and DIOR-R datasets. Visualization of detection results on the DOTA-v1.0 using DP2PNet (FCOS) is shown in Figure 3 and Figure 4. The visualized detection results demonstrate the effectiveness of our method. In addition, the results on the DOTA-v1.0 dataset and the DIOR-R dataset are shown in Table 3 and Table 4.

(1)Results on DOTA-v1.0: As shown in Table 3, DP2PNet achieves 53.82% on the $[eqn]$ by training Rotated FCOS, surpassing point converters such as Point2RBox and PointOBB, which use manually designed hyperparameters. Additionally, our method exhibits competitive performance on the 7- $[eqn]$ compared to the P2RBox method that uses the SAM method as prior information.(2)Results on DIOR-R: As shown in Table 4, DP2PNet achieved 52.14% and 53.61% on mAP50 by training Rotated FCOS and Oriented R-CNN [32], respectively. DP2PNet’s performance surpasses the two-stage alternative, Point-to-HBox-to-RBox (P2BNet + H2RBox-v2), on the $[eqn]$ metric. Moreover, DP2PNet achieves a 21% improvement in performance on the 8- $[eqn]$ metric compared to the point converter PointOBB, and it also demonstrates competitive performance with SAM and fully supervised methods.

To verify the generalization ability across different scenarios, we conduct experiments on the ICDAR2015 [33] dataset, a benchmark for scene text detection. It contains 1500 images with over 5000 text instances annotated with rotated bounding boxes. Experimental results show that DP2PNet with FCOS backbone achieves 51.2% mAP50, outperforming PointOBB (47.8%) and P2RBox (49.5%). The strong cross-scenario generalization of DP2PNet stems from two key designs: the multi-scale consistent noise generator adapts to scale differences between aerial objects and scene text, while the Noise Cross-Constraint module enhances the model’s ability to perceive diverse object shapes in non-remote sensing scenarios. Unlike methods relying on scenario-specific priors, our noise-based prior ensures broader applicability.

4.2.2. Analysis of Performance Difference

On the DOTA dataset, the $[eqn]$ of our method is slightly lower than that of P2RBox (64.84% vs. 67.25%). The main reason is that P2RBox uses SAM to generate high-precision mask priors, which can more accurately locate the edge of the object, especially for small-scale targets such as a small vehicle. The mIoU of our method for a small vehicle is 58.7%, while that of P2RBox is 62.3%. However, our method does not rely on external models such as SAM and has better generalization and efficiency.

On the DIOR-R dataset, our method achieves competitive performance with SAM (53.61% vs. 53.73% in mAP50). This is because the DIOR-R dataset has a large number of large-scale targets, such as airplanes and ships, and our multi-scale consistent noise generator can better adapt to the scale characteristics of these targets, so as to achieve better performance.

4.2.3. Comparison of Method Complexity

To further evaluate the comprehensive performance of our method, we compare the complexity and efficiency of our method with existing methods, as shown in Table 5.

It can be seen from Table 5 that our method has fewer parameters and faster inference speed than P2RBox and PointOBB. At the same time, our method has lower prior dependence and can maintain higher performance under the condition of less training data, which shows that our method has better efficiency and generalization.

4.3. Limitations and Future Directions

Despite promising results, DP2PNet has limitations: it exhibits relatively lower performance on ultra-small objects (pixel-level) due to challenges in noise point coverage matching. For future work, we plan to: (1) Extend to semi-supervised learning scenarios by combining a small number of fully annotated RBBs to further improve accuracy; (2) Adapt to other weak supervision annotation types, such as line annotations and region annotations, by modifying the noise generation and aggregation strategies; (3) Explore integration with transformer-based feature extractors to enhance semantic feature learning. The noise-based prior design enables DP2PNet to generalize to domains beyond computer vision, such as medical image analysis.

5. Ablation Study

In this section, a series of ablation experiments is designed to validate the key components of DP2PNet.

5.1. The Effect of Multi-Scale Map

Table 6 investigates the effect of the first key design, multi-scale mapping, during the diffusion and denoising process in this paper. As shown in Table 6, mapping noisy points to a single receptive field feature map alone leads to varying degrees of performance degradation. This is because the noisy points in a single receptive field cannot accurately indicate the crucial semantic parts of objects of different sizes. For example, the P2 feature map has a small receptive field, which is suitable for small-scale targets, but the mAP50 for large-scale targets, such as a harbor, is only 42.3%. The P4 feature map has a large receptive field, which is suitable for large-scale targets, but the mAP50 for small-scale targets, such as a small vehicle, is only 40.1%. When using multi-scale feature maps (P2 + P3 + P4), the model can adapt to targets of different scales, and the mAP50 for both small and large-scale targets is improved to more than 50%.

5.2. The Effect of Single-Scale Consistent

Table 7 investigates the effect of the second key design in this paper, single-scale consistency. It can be observed that as the number of bags of noise points with the same receptive field, denoted as K, increases, the model’s performance first rises and then saturates at $[eqn]$ . This is because three bags of noise points with the same receptive field are sufficient to cover objects of different shapes well. When $[eqn]$ , the noise point bag may not cover the object completely, resulting in low mAP50 (48.92%). When $[eqn]$ , the coverage of the object is improved, and the mAP50 increases to 55.67%. When $[eqn]$ , the coverage of the object is further improved, and the mAP50 reaches 58.98%. When $[eqn]$ , increasing K further will only add to the computational burden of the model, but will not significantly improve the performance, and the mAP50 even decreases slightly (58.24% when $[eqn]$ ).

5.3. The Effect of Progressive Refinement

Table 8 examines the effect of the third key design in this paper, progressive refinement. As shown in Table 8, the model in this paper can conveniently adjust the refinement stage by modifying the sampling steps during the denoising process. It is observed that the effectiveness of the model gradually improves as the refinement stages increase, reaching a saturation point when the refinement stage number is 5. When the number of refinement stages is 1, the model has not yet fully denoised the noise points, and the pseudo-RBBs have large deviations, resulting in low mAP50 (37.56%). As the number of refinement stages increases to 3, the model further denoises the noise points, and the mAP50 increases to 40.65%. When the number of refinement stages reaches 5, the model has basically completed the denoising process, and the mAP50 reaches 43.15%. When the number of refinement stages exceeds 5, excessive denoising will lead to the loss of useful information, and the mAP50 decreases slightly (42.87% when $[eqn]$ ).

5.4. The Effect of Point Padding Strategy

As mentioned in Section 3.3, we need to fill additional points for the original single-point annotation of each object so that each point’s point bag contains the same number of points N. We have studied different filling strategies in Table 9. It can be seen that the “Uniform” filling strategy (connecting random points uniformly) achieves the best performance, with mAP50 of 43.12%. The “Repeat” filling strategy (duplicating existing points) has poor performance because it cannot expand the coverage of the point bag, resulting in incomplete object coverage. The “Gaussian” filling strategy (filling points according to a Gaussian distribution) has slightly lower performance than the “Uniform” strategy because the Gaussian distribution is concentrated near the original point, and the coverage of the point bag is limited.

5.5. The Effect of Point Annotation Deviation Range

Table 10 shows the impact of different point annotation deviation ranges on generating point labels. It can be seen that because DP2PNet uses a diffusion model for modeling, it is not sensitive to the noise introduced during labeling. When the deviation range increases from 10% to 50%, the mAP50 only decreases by 0.04% (from 43.15% to 43.11%), and the mIOU only decreases by 0.12% (from 58.98% to 58.86%). This is because the diffusion model has strong noise robustness, and the multi-scale consistent noise generator can adapt to the deviation of point annotations by adjusting the noise distribution.

5.6. The Effect of Noise Level

To verify the noise robustness of DP2PNet, we designed an experiment on the effect of noise level. We set 5 noise levels (noise standard deviation $[eqn]$ ) and compared the performance of our method with PointOBB and P2RBox. The results are shown in Table 11.

It can be seen from Table 11 that with the increase of noise level, the performance of all methods decreases, but the performance attenuation rate of our method is significantly lower than that of PointOBB and P2RBox. When the noise standard deviation reaches 0.9, the performance attenuation rate of our method is only 0.90%, while that of PointOBB and P2RBox is 21.83% and 15.37%, respectively. This shows that our method has strong noise robustness, which is due to the fact that the diffusion model itself is a noise modeling process, and the multi-scale consistent noise generator can adapt to different noise levels by adjusting the noise distribution.

5.7. The Effect of Target Type

To verify the adaptability of DP2PNet to different types of targets, we selected three types of typical targets in the DOTA dataset (small-scale: Small Vehicle, large-scale: Harbor, irregular shape: Baseball Diamond) and compared the performance of our method with existing methods. The results are shown in Table 12.

It can be seen from Table 12 that our method has good adaptability to targets of different types. For small-scale targets (Small Vehicle), although the mAP50 of our method is lower than that of P2RBox, it is higher than that of PointOBB. For large-scale targets (harbor), the mAP50 of our method is higher than that of PointOBB and close to that of P2RBox. For irregular shape targets (Baseball Diamond), the mAP50 of our method is higher than that of PointOBB and close to that of P2RBox. This shows that our method can adapt to the characteristics of different types of targets through the multi-scale consistent noise generator and the Semantic Key Point Aggregator.

5.8. Cross-Dataset Generalization Experiment

To verify the cross-dataset generalization ability of DP2PNet, we trained the model on the DOTA dataset and tested it directly on the DIOR-R dataset without fine-tuning. We compared the performance of our method with PointOBB and P2RBox. The results are shown in Table 13.

It can be seen from Table 13 that our method has better cross-dataset generalization ability. The performance retention rate of our method is 90.0%, which is higher than that of PointOBB (80.3%) and P2RBox (78.5%). This is because our method uses noise as the only prior, which avoids the over-reliance on dataset-specific prior information (such as manually designed scales or external model masks) in existing methods, thus enhancing the generalization across different datasets.

6. Conclusions

In this paper, we propose DP2PNet, which is the first diffusion model-based framework for single-point supervised oriented object detection, addressing the key limitations of existing methods. Three core innovations drive its performance: The multi-scale consistent noise generator replaces manual or external model priors with Gaussian noise, eliminating domain-specific constraints; The Noise Cross-Constraint module enables multi-scale object perception through receptive field matching and object coverage fusion; The Semantic Key Point Aggregator generates pseudo-RBBs via convex hulls of graph-convolution-aggregated semantic points, avoiding direct angle prediction. Experimental results show that DP2PNet achieves 53.82% mAP50 on DOTA-v1.0 and 53.61% on DIOR-R, outperforming PointOBB by over 13% and demonstrating competitive performance with P2RBox and SAM-based methods. Compared to state-of-the-art methods, DP2PNet has fewer parameters and faster inference speed (22 FPS vs. 18 FPS for PointOBB). Future work will focus on three directions: integrating transformer architectures to enhance feature extraction, extending to semi-supervised and few-shot learning scenarios, and adapting to diverse weak supervision annotation types.