DFIR-DETR: Frequency-Domain Iterative Refinement and Dynamic Feature Aggregation for Small Object Detection

TL;DR

DFIR-DETR introduces a transformer-based small object detector that enhances feature aggregation and detail preservation through frequency-domain refinement and adaptive attention, achieving state-of-the-art results with fewer parameters.

Contribution

It proposes three novel modules—DCFA, DFPN, and FIRC3—that improve small object detection by focusing attention, preserving high-frequency details, and maintaining norm stability.

Findings

Achieves 92.9% mAP50 on NEU-DET

Achieves 51.6% mAP50 on VisDrone

Operates with only 11.7M parameters and 41.2 GFLOPs

Abstract

Small object detection in complex scenes exposes a fundamental tension in neural network design: backbone attention distributes computation uniformly regardless of content, pyramid necks inflate activation magnitudes during upsampling without norm compensation, and bottleneck convolutions progressively smooth high-frequency edge components through accumulated spatial filtering. To address each failure mode, we propose DFIR-DETR, a transformer-based detector built around three principled contributions: Dynamic Content-Feature Aggregation (DCFA), which concentrates self-attention on structurally complex regions via input-adaptive Top-K sparsification, reducing complexity from O(N2) to O(NK); a Dynamic Feature Pyramid Network (DFPN), which establishes norm-preserving upsampling and explicit spatial detail recovery through dual-path convolution; and a Frequency-domain Iterative Refinement module (FIRC3), which formulates feature aggregation as a constrained optimisation problem in the spectral domain, directly preserving high-frequency boundary components that spatial operations cannot retain. On NEU-DET and VisDrone, DFIR-DETR achieves 92.9% and 51.6% mAP50 with only 11.7M parameters and 41.2~GFLOPs, demonstrating consistent gains across two qualitatively different detection domains.