Microsoft COCO Captions: Data Collection and Evaluation Server

TL;DR

This paper introduces the Microsoft COCO Caption dataset with over 330,000 images and 1.5 million captions, along with an evaluation server that scores caption quality using multiple metrics to standardize automatic captioning evaluation.

Contribution

It presents a large-scale caption dataset and an evaluation server, enabling consistent benchmarking of image captioning algorithms.

Findings

Dataset contains over 330,000 images with 1.5 million captions.

Evaluation server supports multiple metrics like BLEU, METEOR, ROUGE, CIDEr.

Provides standardized platform for captioning algorithm assessment.

Abstract

In this paper we describe the Microsoft COCO Caption dataset and evaluation server. When completed, the dataset will contain over one and a half million captions describing over 330,000 images. For the training and validation images, five independent human generated captions will be provided. To ensure consistency in evaluation of automatic caption generation algorithms, an evaluation server is used. The evaluation server receives candidate captions and scores them using several popular metrics, including BLEU, METEOR, ROUGE and CIDEr. Instructions for using the evaluation server are provided.