Learning Where to Embed: Noise-Aware Positional Embedding for Query Retrieval in Small-Object Detection

TL;DR

This paper introduces HELP, a noise-aware positional embedding framework for small-object detection that improves efficiency and accuracy by selectively embedding positional information and filtering background noise.

Contribution

The paper proposes a novel heatmap-guided embedding method that enhances query retrieval and reduces model complexity in small-object detection tasks.

Findings

Achieves 59.4% parameter reduction with maintained accuracy.

Reduces decoder layers from eight to three.

Improves small-object detection performance across benchmarks.

Abstract

Transformer-based detectors have advanced small-object detection, but they often remain inefficient and vulnerable to background-induced query noise, which motivates deep decoders to refine low-quality queries. We present HELP (Heatmap-guided Embedding Learning Paradigm), a noise-aware positional-semantic fusion framework that studies where to embed positional information by selectively preserving positional encodings in foreground-salient regions while suppressing background clutter. Within HELP, we introduce Heatmap-guided Positional Embedding (HPE) as the core embedding mechanism and visualize it with a heatbar for interpretable diagnosis and fine-tuning. HPE is integrated into both the encoder and decoder: it guides noise-suppressed feature encoding by injecting heatmap-aware positional encoding, and it enables high-quality query retrieval by filtering background-dominant embeddings via a gradient-based mask filter before decoding. To address feature sparsity in complex small targets, we integrate Linear-Snake Convolution to enrich retrieval-relevant representations. The gradient-based heatmap supervision is used during training only, incurring no additional gradient computation at inference. As a result, our design reduces decoder layers from eight to three and achieves a 59.4% parameter reduction (66.3M vs. 163M) while maintaining consistent accuracy gains under a reduced compute budget across benchmarks. Code Repository: https://github.com/yidimopozhibai/Noise-Suppressed-Query-Retrieval