Integrating Diverse Assignment Strategies into DETRs
Yiwei Zhang, Jin Gao, Hanshi Wang, Fudong Ge, Guan Luo, Weiming Hu, Zhipeng Zhang

TL;DR
This paper introduces LoRA-DETR, a lightweight framework that integrates diverse assignment strategies into DETR detectors, improving performance by leveraging varied supervision without increasing inference complexity.
Contribution
The paper reveals that diversity in assignment strategies, rather than quantity, boosts DETR performance and proposes a scalable, parameter-efficient method to incorporate this diversity during training.
Findings
Achieves state-of-the-art results on multiple benchmarks.
Diverse assignment strategies outperform single strategies.
No additional inference cost due to auxiliary training branches.
Abstract
Label assignment is a critical component in object detectors, particularly within DETR-style frameworks where the one-to-one matching strategy, despite its end-to-end elegance, suffers from slow convergence due to sparse supervision. While recent works have explored one-to-many assignments to enrich supervisory signals, they often introduce complex, architecture-specific modifications and typically focus on a single auxiliary strategy, lacking a unified and scalable design. In this paper, we first systematically investigate the effects of ``one-to-many'' supervision and reveal a surprising insight that performance gains are driven not by the sheer quantity of supervision, but by the diversity of the assignment strategies employed. This finding suggests that a more elegant, parameter-efficient approach is attainable. Building on this insight, we propose LoRA-DETR, a flexible and…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsAdvanced Neural Network Applications · Domain Adaptation and Few-Shot Learning · Machine Learning and Data Classification
