Self-supervised structured object representation learning

TL;DR

This paper introduces a self-supervised learning method that captures structured scene representations across multiple scales, improving object detection performance especially with limited labeled data.

Contribution

It proposes a novel ProtoScale module for hierarchical visual representation learning that preserves scene context and enhances dense prediction tasks.

Findings

Outperforms state-of-the-art methods in object detection

Effective with limited annotated data

Improves dense prediction tasks

Abstract

Self-supervised learning (SSL) has emerged as a powerful technique for learning visual representations. While recent SSL approaches achieve strong results in global image understanding, they are limited in capturing the structured representation in scenes. In this work, we propose a self-supervised approach that progressively builds structured visual representations by combining semantic grouping, instance level separation, and hierarchical structuring. Our approach, based on a novel ProtoScale module, captures visual elements across multiple spatial scales. Unlike common strategies like DINO that rely on random cropping and global embeddings, we preserve full scene context across augmented views to improve performance in dense prediction tasks. We validate our method on downstream object detection tasks using a combined subset of multiple datasets (COCO and UA-DETRAC). Experimental results show that our method learns object centric representations that enhance supervised object detection and outperform the state-of-the-art methods, even when trained with limited annotated data and fewer fine-tuning epochs.

Tables

Table 1. Table 1: Detection performance under varying proportions of labeled training data with 10 fine-tuning epochs.

Training set	Methods	$AP$	$A{P}_s$	$A{P}_m$	$A{P}_l$	$AR$	$A{R}_s$	$A{R}_m$	$A{R}_l$
10%	DINOv2	20.1	$6.5$	$17.$2	29.58	$57.9$	$31.1$	$52.5$	73.0
10%	ProtoScale	$19.9$	10.4	17.8	$22.5$	58.2	36.7	53.3	$70.2$
30%	DINOv2	$37.5$	$13.14$	$33.4$	50.1	$63.6$	$39.4$	$60.0$	$78.1$
30%	ProtoScale	41.0	21.1	39.3	$48.2$	68.4	52.1	66.5	79.2
50%	DINOv2	$50.7$	$17.9$	$46.6$	65.8	$68.9$	$41.9$	$66.0$	$83.8$
50%	ProtoScale	57.1	31.2	56.6	$65.3$	75.2	59.7	74.2	85.2
70%	DINOv2	$50.6$	$18.11$	$46.8$	$65.15$	$69.10$	$44.26$	$66.0$	$83.9$
70%	ProtoScale	58.7	31.0	57.8	68.0	76.5	61.4	75.9	86.6
100%	DINOv2	$63.7$	$36.2$	$66.6$	$70.0$	$76.6$	$58.4$	$77.6$	91.5
100%	ProtoScale	68.3	47.4	68.7	76.1	79.8	64.8	79.3	$88.9$

Equations

$$A{s}_k=\text{softmax}\left(\frac{S_k^{\top }\cdot F_k}{\tau }\right),$$

$$A{i}_k=\sum _{i=1}^{Np}{\text{softmax}}_i\left(\frac{\Vert S_k\Vert \cdot \Vert I_k\Vert }{\tau }\right)A{s}_k,$$

Full text

1 Introduction

Large language models such as GPT [16], BERT [6], and LLaMA [18] have demonstrated remarkable performance across a wide range of tasks, largely due to their ability to learn rich representations directly from row data. A key factor behind their success is the use of self-supervised learning (SSL), where models are trained to predict parts of their input from other parts, without relying on human annotations. The computer vision community has been inspired by SSL paving the way for more robust, unbiased, and generalizable systems that can scale to open world data and complex scenarios.

SSL approaches such as DINO [3], iBOT [24], and BEiT [1] have extended these ideas to image understanding. Most of these methods rely on large Transformer based architectures (VIT [7]) and learn global representations by solving auxiliary tasks such as predicting similarities between augmented views. These models have proven effective for image classification and matching, however, they often struggle to capture the compositional structure of objects in scenes. MMethods like CutLER [19] typically identify a single dominant foreground region but struggle to isolate multiple objects without costly iterative training. Slot Attention [14] or SlotCon [22] aim to produce structured scene representations through implicit grouping mechanisms, but lacks explicit hierarchical organization and instance level decomposition.

Current limitations are twofold. First, their design tends to produce representations of entire scenes, with limited capacity for structured object understanding. Yet an image is far more than a single label, it contains objects, parts, and spatial relationships. A meaningful visual representation should reflect this multi level structure and represent features at local, intermediate, and global scales. Second, fine-tuning these models for dense prediction tasks often demands substantial computational resources and extensive annotated data, making deployment challenging in resource constrained settings.

We introduce an SSL approach based on a new module called ProtoScale designed to overcome these limitations by progressively constructing a structured visual representation that combines semantic abstraction, instance separation, and hierarchical organization. Our approach builds on the intuition that an effective representation should be able to: (i) group pixels based on semantic meaning, (ii) identify distinct parts of instances, and (iii) organize them coherently to represent complete objects. To test this hypothesis, we focus on the task of object detection, which offers a good balance between local and global scene complexity for interpretation. The objective is to capture complexity of visual scenes without relying on manual annotations, while supporting hierarchical object learning at multiple scales. Unlike state-of-the-art (SOTA) methods that rely on disruptive cropping strategies, we preserve the full scene context to ensure instance consistency while maintaining spatial structure. In addition, we adopt a lightweight hybrid encoder to reduce computational cost and improve feature scalability. We assess the representation learning ability of the proposed model by conducting downstream evaluations on the object detection task, using a combined subset of multiple datasets including COCO [13] and UA-DETRAC [21]. We challenge our method under conditions where annotations are limited and training does not rely on large scale computational resources. Unlike traditional downstream task pipelines based on Mask R-CNN, which rely heavily on post processing steps such as Non Maximum Suppression (NMS) and are often less efficient in real world applications, we adopt a DETR-like architecture [2], which provides a fully end to end formulation without NMS, and better aligns with our goal of evaluating structured, object centric representations in a simplified and deployable detection setting. Our method achieves strong performance on COCO and UA-DETRAC, even when trained for only $10$ epochs and with just $50$% of the annotated training data.

In summary, the main contributions of this paper are as follows: (1) SSL approach that combines semantic grouping, instance level clustering, and hierarchical reasoning to build structured visual representations without any annotations. (2) ProtoScale, a modular grouping module operating at multiple spatial scales, capable of organizing pixels into instances and instances into structured groups. (3) Preserving full scene context across views by avoiding disruptive cropping, improving consistency in dense prediction tasks. (4) Leveraging a lightweight hybrid CNN/Transformer encoder, adapted from RT-DETR [23], to balance feature precision and reduce computational overhead. (5) Validation of the proposed approach through downstream object detection experiments by comparing it with SOTA methods from the literature and evaluating it under realistic constraints, using a small annotated dataset and a limited number of training epochs.

2 Related work

Most SSL methods in computer vision relies on contrastive learning, where the objective is to bring closer the representations of two different views of the same image. The approaches in SimCLR [4] and MoCo [9], typically use two networks, one for the original image and the other with its transformed one, aim to maximize their similarity in latent space.

DINO [3] extends this paradigm by introducing a teacher–student framework, where the teacher model stabilizes the training of the student via momentum updates. This framework leads to the spontaneous emergence of structured visual patterns such as foreground and background regions. Several variants such as iBOT [24], and TEC [8] further enrich this contrastive logic by incorporating multi view representations and dense patch level objectives. While these methods have driven significant progress in representation learning, they remain limited in their ability to structure objects in images. Their primary goal is to encode an entire image as a global vector, which is suitable for image level tasks like classification, but fall apart for dense prediction.

Some recent efforts have tackled the problem of object discovery without supervision by leveraging perceptual grouping cues, through clustering as in MOST [17], or an iterative supervised learning loop in the case of CutLER [19]. While effective at discovering salient regions, these approaches remain limited by the quality of the initial pseudo labels and often focus on a single dominant object per image.

In parallel, dense SSL methods have emerged, aiming to align features at the pixel or region level, like DetCon [11], and SoCo [20]. These methods learn features that preserve spatial coherence and encourage local consistency across regions within an image. Other works use latent slots as intermediate representations to structure objects in scenes. Slot Attention [14] uses an iterative attention mechanism to assign pixels to abstract instances. SlotCon [22] combines semantic grouping with contrastive learning to extract salient, object centric representations. Inspired by SlotCon, we extend its formulation by modifying the semantic grouping process and explicitly modeling hierarchical relationships between instances.

Most recent SSL methods rely on pure Transformer based architectures, especially Vision Transformers (ViTs), which have shown strong performance on classification tasks. However, ViTs are computationally expensive in both memory and training time. Moreover, their patch based processing scheme fragments the input and removes local spatial continuity, limiting their ability to capture small or occluded objects. In this work, we adopt a CNN/Transformer hybrid encoder proposed by RT-DETR [23] to reduce computational cost while preserving fine spatial cues, which is particularly beneficial for unsupervised dense prediction.

While recent SSL methods have made impressive progress, they remain incomplete when it comes to producing structured image representations. Contrastive methods yield effective global or local representation but lack grouping capabilities. Latent slot based models offer encouraging directions but still fall short in terms of hierarchy, modularity, and generalization. We addresses this gap by proposing a modular, hierarchical grouping mechanism based on shared visual prototypes, integrated into contrastive learning framework.

3 Object-level contrastive learning

The proposed method is based on the student–teacher framework (see Fig. 1), commonly used in self-supervised learning. Both networks share the same encoder architecture but differ in their input processing and training roles. Unlike SOTA methods, the student is fed with multiple semantically diverse yet spatially coherent augmented views of each input image, while the teacher receives a single global augmented view with stronger spatial transformations and serves as a stable label generator. Its parameters are updated using an exponential moving average (EMA) of the student weights.

Training encourages the student predictions across all views to align with the teacher output using three loss terms, promoting view consistent grouping. These losses are tightly integrated with our proposed module, ProtoScale, which groups elements of the scene based on their semantic meaning, instance assignment, and hierarchical relations. This module favors a cross perspective understanding of object structure and enables scene dense parsing. To capture visual regularities at different levels of abstraction, both models share a multi scale encoder. This encoder extracts feature maps at three spatial resolutions, enabling ProtoScale to operate simultaneously at fine, intermediate, and global scales. This design helps the model discover small object parts, full object, and global scene in a unified feature representation.

3.1 Multi-view learning

Our approach relies on an asymmetric multi view learning strategy to encourage consistent grouping across appearance variations. The teacher processes a single globally augmented image obtained through a sequence of spatial transformations, including random cropping, zoom out, horizontal flipping, and photometric distortions. This image serves as the stable target for training and as the base input from which the student views are generated. In contrast, the student is exposed to four distinct views of the same input, sampled from a different transformation pipeline. These augmentations are designed to be spatially coherent, preserving object layout and relative positioning, while introducing sufficient appearance variability. Specifically, they include Gaussian blur, random spatial masking, and color jitter. Unlike SOTA methods that rely heavily on random cropping strategies which disrupt the spatial distribution of objects and degrade dense prediction performance, we adopt a strategy that preserves the full scene context across views. This design choice promotes instance consistency without compromising the underlying spatial structure.

3.2 Hybrid CNN-Transformer pyramid encoder

Unlike previous SSL methods, we employ a hybrid encoder architecture inspired by RT-DETR. The encoder processes multi scale features extracted from a ResNet [10] backbone. The lowest resolution feature map is passed through a full self-attention transformer, capturing long range spatial dependencies in the image and providing a coarse indication of where the model should focus. This global attention serves as guidance to identify regions of interest in the image. The resulting low resolution attention is then progressively fused with intermediate and high resolution convolution features through a combination of top down and bottom up fusion paths. This stage relies on the local precision of CNN features to determine what is present in those regions. This efficient hybrid design enables accurate dense representation while reducing the computational cost associated with full resolution attention.

3.3 ProtoScale: a multi-level grouping modules

Once the features are extracted from the encoders, they are processed by our proposed grouping module, ProtoScale (Fig. 2). This module consists of three sequential components. First, a semantic clustering stage is applied independently to each feature map, where spatial attention maps are generated using learnable prototypes (cluster centers). Second, the resulting semantic prototypes from the entire batch are aggregated into a shared set of instance level prototypes, enabling consistent grouping of object-like regions across different samples. Finally, a lightweight hierarchical module infers relationships between these instances, producing a hierarchical grouping that captures object–part structures at the batch level.

3.3.1 Semantic grouping

At each output scale $k$ of the encoder, spatial features are projected and fed into a semantic grouping module based on prototypes attention. Inspired by SlotCon, we compute dot product attention between $256$ learnable semantic prototypes $S_{k}\in\mathcal{R}^{256}$ and spatial features $F_{k}$ as follows:

[TABLE]

with $\tau>0$ temperature parameter that control the sharpness of the output distribution. Each prototype $S_{k}$ aggregates features through weighted attention to form semantic queries. These attention maps $As_{k}$ highlight spatial groupings based on shared appearance features.

Unlike SlotCon, we integrate this grouping into a dense prediction pipeline by applying a supervised loss between the student and teacher attention maps at multiple scales. Furthermore, we introduce a centered Gaussian prior ($\mu=0.5,\sigma=0.7$) applied to the attention maps to reduce collapse and mitigate the bias toward generic image representations, encouraging focus on local structure instead. Inspired by the work of [12], we also incorporate a set of auxiliary prototypes $R$ (see Fig. 2). designed to absorb noisy activations, these are excluded from the loss to stabilize learning. This setup enables the model to learn interpretable, multi level semantic decompositions.

3.3.2 Instance grouping via shared latent prototypes

While semantic grouping isolates regions, it fail in crowded or occluded scenes. We address this by introducing a differentiable instance grouping module based on shared latent prototypes $I_{k}$, learned across the batch.

Each slot $S_{k}$ is softly assigned to the $32$ learnable parameter ($I_{k}\in\mathcal{R}^{256}$). The resulting assignment weights are then used to scale the corresponding attention map $As_{k}$ before summation over all semantic prototypes ($Np$) :

[TABLE]

with $\tau>0$ temperature parameter. These instance attention reflect coherent object level regions. The sub-module is trained with consistency and sparsity regularization, encouraging specialization and spatial diversity among instances prototypes.

3.3.3 Hierarchical object understanding through prototype affinities

Beyond instances level attentions, ProtoScale also capture object relationships by modeling the affinity between instance prototypes. For each scale $k$, a relation matrix ($H_{k}=\sigma(\text{MLP}\left(I_{k})\right)$) between instances prototypes $I_{k}$ is predicted using a shallow multilayer perceptron ($MLP$), followed by sigmoid activation ($\sigma$). This matrix is used to define hierarchical attention ($Ah_{k}=H_{k}\cdot Ai_{k}$). By thresholding low affinity pairs at $0.5$, the model learns to merge overlapping or related instances, capturing part-whole and object group structures. This module enriches the learned representation with scene level structure and abstraction.

3.4 Self-supervised objective

The student is trained to match the teacher outputs using a weighted objective composed of three terms, a semantic loss, an instance level loss, and a hierarchical loss. The overall loss function is defined as : $\mathcal{L}_{\text{total}}=\lambda_{\text{sem}}\mathcal{L}_{\text{sem}}+\lambda_{\text{inst}}\mathcal{L}_{\text{inst}}+\lambda_{\text{hier}}\mathcal{L}_{\text{hier}}$, where $(\lambda_{\text{sem}},\lambda_{\text{inst}},\lambda_{\text{hier}})=(1,2,1)$ yields the best results in our experiments. Each loss includes a KL divergence term computed between the corresponding attention maps of the student and the teacher. The KL divergence was chosen to encourage the student model to match the entire probability distribution rather than focusing only on the highest-probability regions. All modules operate at three spatial scales, enabling rich object-centric reasoning across fine, intermediate, and global resolutions.

4 Experimental setting

In this section, we first describe the baselines used in our evaluations, then present the datasets, and finally explain the evaluation setup.

4.1 Baselines

Self-supervised encoders are typically evaluated through downstream fine-tuning on detection or segmentation tasks, often using Mask R-CNN. However, in this work, we adopt the RT-DETR framework, which offers an end to end detection pipeline without relying on post processing steps such as non maximum suppression. Moreover, RT-DETR achieves significantly better performance compared to earlier methods like Mask R-CNN. Unfortunately, RT-DETR does not support instance segmentation, which prevents us from evaluating segmentation performance. Therefore, we focus exclusively on object detection as the downstream task. We select a set of strong SSL baselines from the literature: DINOv1 [3], DINOv2 [15], iBOT [24], TEC [8] and SlotCon [22]. For each method, we use the encoder backbone and attach it with a shared projection module that outputs multi scale features. These encoders are integrated into our student–teacher method for a fair comparison under the same training configurations. We also report results for the fully supervised RT-DETR [23] model as an reference.

4.2 Dataset

We conduct experiments on two datasets selected for their complementary characteristics offering high scene diversity and high object density, respectively for COCO and UA-DETRAC. To simulate real world scenarios, we redefine the original class labels into a fine-grained taxonomy of 17 classes. We extract subsets from each dataset by selecting only images containing at least two of these classes, resulting in $39429$ training and $4500$ validation images from COCO, and $10539$ training and $1055$ validation images from UA-DETRAC. For a combined total of $50022$ training and $5555$ validation images.

4.3 Evaluation setups

The core principle of SSL is to enable the fine-tuning of downstream models with limited labeled data and within a small number of epochs, while still achieving strong generalization. Following standard practice in SSL, we assess the representation capability of the pre-trained model by using it as the backbone for a downstream detection task. Instead of Mask R-CNN, we adopt RT-DETR and fine-tune all layers end to end using different SSL pretrained backbones. To simulate a realistic low resource scenario, we begin by fine-tune the detection model for a varying number of epochs and compare its performance with related methods (see Fig. 3). We then evaluate the impact of using different portions of the training data (see Tab. 1), while always testing on the full validation set. These experiments reflect challenging conditions in which annotated data and computation resources are limited, typical of real world environments where access to high computing infrastructure and large annotated datasets for fine-tuning is often unavailable. All experiments are conducted on dual NVIDIA RTX A6000.

5 Results and discussion

In this section, we evaluate the performance of our SSL model through a downstream object detection task. We compare ProtoScale with several SOTA SSL methods on a combined subset of multiple datasets. Performance is reported using the standard COCO detection metrics. $AP$ and $AR$ denote the mean Average Precision and mean Average Recall, respectively, over IoU thresholds $[0.50:0.95]$. $AP_{50}$ and $AP_{75}$ correspond to $AP$ at $IoU=0.50$ and $IoU=0.75$, respectively. Results are also reported by object scale: small ($AP_{s}$, $<32^{2}$ px), medium ($AP_{m}$, $32^{2}\leq$ area $<96^{2}$ px), and large ($AP_{l}$, $\geq 96^{2}$ px). $AR$ is defined analogously under the same settings.

5.1 Sensitivity to fine-tuning epochs

In Fig. 3, we present the main results using models pre-trained on COCO [13], ImageNet-1k [5] and LVD-142M [15] datasets. Specifically, we include SlotCon pre-trained for 800 epochs on COCO, DINOv1 with ViT Small trained for 300 epochs on ImageNet-1K, DINOv2 with ViT Small trained for 300 epochs on LVD-142M, iBOT with ViT Small trained for 800 epochs on ImageNet-1K, TEC with ViT-base trained for 800 epochs on ImageNet-1K, and Protoscale pre-trained for 35 epochs on COCO. We also include the RT-DETR baseline with random initialization as a supervised reference. All methods are fine-tuned on the combined dataset comprising COCO and UA-DETRAC. ProtoScale significantly outperforms current SOTA SSL approaches, as well as the supervised RT-DETR with random initialization, consistently achieving higher performance even compared to models pre-trained on large scale datasets such as ImageNet-1K and LVD-142M. Notably, our method demonstrates strong performance on small and medium objects, even with fewer fine-tuning epochs. For example, SlotCon requires up to 800 epochs of pre-training to achieve competitive results. However, DINOv2 slightly outperforms our method on large objects, likely due to the strong generalization capabilities of transformer based architectures trained on a massive dataset of 142 million image.

5.2 Influence of labeled data availability

In Tab. 1, we further analyze the impact of limited labeled data on fine-tuning performance for the object detection task. We select different proportions of the training set (10%, 30%, 50% and 70%) to simulate low resource scenarios. We then compare the two best performing methods from the previous experiment (ProtoScale and DINOv2) to evaluate their robustness under varying data availability. Although the overall performance decreases compared to fine-tuning on 100% of the training data, the results demonstrate that our approach requires less labeled data to achieve competitive performance. Notably, ProtoScale consistently outperforms DINOv2 on small and medium objects, while DINOv2 performs slightly better on large objects. Although the difference remains small and the DINOv2 encoder (ViT Small) contains approximately 21 million parameters, whereas our model uses only 12 million.

6 Conclusion

This work present a SSL approach that constructs structured visual representations through semantic grouping, instance level clustering, and hierarchical reasoning. Our proposed module, ProtoScale, operates at multiple spatial scales to organize scenes into semantically coherent and object centric components. By avoiding disruptive cropping and preserving full scene context across views, our method enhances consistency in dense prediction tasks. We validated our approach through downstream object detection experiments on combined subsets of COCO and UA-DETRAC, demonstrating strong generalization under realistic low resource constraints. ProtoScale consistently outperforms SOTA SSL methods, including those trained on large scale datasets, while using fewer parameters and requiring fewer training epochs. Future work will investigate the effectiveness of our method on larger datasets and explore its applicability to other dense prediction tasks. We also aim to design a SSL decoder to enhance the interpretability of the learned representations in a human-understandable manner.