Microsoft COCO: Common Objects in Context

TL;DR

The paper introduces the Microsoft COCO dataset, a large-scale collection of complex scene images with detailed object annotations, designed to improve object recognition and scene understanding in computer vision.

Contribution

It provides a new, extensive dataset with 2.5 million labeled instances across 328,000 images, featuring detailed annotations and baseline detection results.

Findings

The dataset contains 91 object categories recognizable by a 4-year-old.

Extensive crowd-sourced annotations enable precise object localization.

Baseline detection performance demonstrates the dataset's utility for advancing research.

Abstract

We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding. This is achieved by gathering images of complex everyday scenes containing common objects in their natural context. Objects are labeled using per-instance segmentations to aid in precise object localization. Our dataset contains photos of 91 objects types that would be easily recognizable by a 4 year old. With a total of 2.5 million labeled instances in 328k images, the creation of our dataset drew upon extensive crowd worker involvement via novel user interfaces for category detection, instance spotting and instance segmentation. We present a detailed statistical analysis of the dataset in comparison to PASCAL, ImageNet, and SUN. Finally, we provide baseline performance analysis for bounding box and segmentation detection results using a Deformable Parts Model.