YOLO-Master: MOE-Accelerated with Specialized Transformers for Enhanced Real-time Detection

TL;DR

YOLO-Master introduces instance-conditional adaptive computation using a specialized MoE framework to improve real-time object detection accuracy and efficiency, especially in complex scenes, while maintaining fast inference speeds.

Contribution

It proposes a novel adaptive computation framework for YOLO-like models using ES-MoE blocks guided by a dynamic routing network, enhancing detection performance and resource allocation.

Findings

Achieves 42.4% AP on MS COCO with 1.62ms latency.

Outperforms YOLOv3-N by +0.8% mAP and is 17.8% faster.

Excels particularly in dense, complex scenes.

Abstract

Existing Real-Time Object Detection (RTOD) methods commonly adopt YOLO-like architectures for their favorable trade-off between accuracy and speed. However, these models rely on static dense computation that applies uniform processing to all inputs, misallocating representational capacity and computational resources such as over-allocating on trivial scenes while under-serving complex ones. This mismatch results in both computational redundancy and suboptimal detection performance. To overcome this limitation, we propose YOLO-Master, a novel YOLO-like framework that introduces instance-conditional adaptive computation for RTOD. This is achieved through a Efficient Sparse Mixture-of-Experts (ES-MoE) block that dynamically allocates computational resources to each input according to its scene complexity. At its core, a lightweight dynamic routing network guides expert specialization during training through a diversity enhancing objective, encouraging complementary expertise among experts. Additionally, the routing network adaptively learns to activate only the most relevant experts, thereby improving detection performance while minimizing computational overhead during inference. Comprehensive experiments on five large-scale benchmarks demonstrate the superiority of YOLO-Master. On MS COCO, our model achieves 42.4% AP with 1.62ms latency, outperforming YOLOv13-N by +0.8% mAP and 17.8% faster inference. Notably, the gains are most pronounced on challenging dense scenes, while the model preserves efficiency on typical inputs and maintains real-time inference speed. Code will be available.