Roboflow 100: A Rich, Multi-Domain Object Detection Benchmark

TL;DR

The paper introduces RF100, a comprehensive multi-domain object detection benchmark with 100 datasets across diverse imagery types, designed to evaluate model generalization beyond traditional datasets.

Contribution

It presents RF100, a large, diverse, multi-domain benchmark dataset derived from over 90,000 datasets, to assess object detection models' generalizability in real-world scenarios.

Findings

RF100 includes 224,714 images and 805 classes across 7 domains.

Provides a new standard for evaluating model generalization.

Enables benchmarking on diverse, real-world datasets.

Abstract

The evaluation of object detection models is usually performed by optimizing a single metric, e.g. mAP, on a fixed set of datasets, e.g. Microsoft COCO and Pascal VOC. Due to image retrieval and annotation costs, these datasets consist largely of images found on the web and do not represent many real-life domains that are being modelled in practice, e.g. satellite, microscopic and gaming, making it difficult to assert the degree of generalization learned by the model. We introduce the Roboflow-100 (RF100) consisting of 100 datasets, 7 imagery domains, 224,714 images, and 805 class labels with over 11,170 labelling hours. We derived RF100 from over 90,000 public datasets, 60 million public images that are actively being assembled and labelled by computer vision practitioners in the open on the web application Roboflow Universe. By releasing RF100, we aim to provide a semantically diverse, multi-domain benchmark of datasets to help researchers test their model's generalizability with real-life data. RF100 download and benchmark replication are available on GitHub.