YOLOBench: Benchmarking Efficient Object Detectors on Embedded Systems
Ivan Lazarevich, Matteo Grimaldi, Ravish Kumar, Saptarshi, Mitra, Shahrukh Khan, Sudhakar Sah

TL;DR
YOLOBench provides a comprehensive benchmark of over 550 YOLO-based object detection models across various datasets and embedded hardware, revealing insights into accuracy-latency trade-offs and the effectiveness of zero-cost estimators.
Contribution
This paper introduces YOLOBench, a large-scale benchmark for YOLO models on embedded systems, and evaluates the effectiveness of zero-cost accuracy estimators in predicting optimal models.
Findings
Modern YOLO architectures achieve good accuracy-latency trade-offs.
Zero-cost estimators can predict Pareto-optimal models, with some outperforming simple baselines.
Older models like YOLOv3 and YOLOv4 remain competitive with newer architectures.
Abstract
We present YOLOBench, a benchmark comprised of 550+ YOLO-based object detection models on 4 different datasets and 4 different embedded hardware platforms (x86 CPU, ARM CPU, Nvidia GPU, NPU). We collect accuracy and latency numbers for a variety of YOLO-based one-stage detectors at different model scales by performing a fair, controlled comparison of these detectors with a fixed training environment (code and training hyperparameters). Pareto-optimality analysis of the collected data reveals that, if modern detection heads and training techniques are incorporated into the learning process, multiple architectures of the YOLO series achieve a good accuracy-latency trade-off, including older models like YOLOv3 and YOLOv4. We also evaluate training-free accuracy estimators used in neural architecture search on YOLOBench and demonstrate that, while most state-of-the-art zero-cost accuracy…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAdvanced Neural Network Applications · Adversarial Robustness in Machine Learning · COVID-19 diagnosis using AI
MethodsBNB Customer Service Number +1-833-534-1729 · You Only Look Once · *Communicated@Fast*How Do I Communicate to Expedia? · 1x1 Convolution · (TravEL!!Guide)How Do I File a Claim with Expedia? · Residual Connection · Feature Pyramid Network · Grid Sensitive · Global Average Pooling · CutMix
