TRECVID 2019: An Evaluation Campaign to Benchmark Video Activity Detection, Video Captioning and Matching, and Video Search & Retrieval

TL;DR

TRECVID 2019 is a comprehensive evaluation campaign that benchmarks progress in video activity detection, captioning, matching, and search through open, metrics-based assessments involving international research teams.

Contribution

This paper introduces the TRECVID 2019 evaluation framework, tasks, data, and measures, continuing a 19-year effort to benchmark video analysis technologies.

Findings

27 teams participated across four tasks

Evaluation metrics facilitated progress tracking

Benchmark results inform future research directions

Abstract

The TREC Video Retrieval Evaluation (TRECVID) 2019 was a TREC-style video analysis and retrieval evaluation, the goal of which remains to promote progress in research and development of content-based exploitation and retrieval of information from digital video via open, metrics-based evaluation. Over the last nineteen years this effort has yielded a better understanding of how systems can effectively accomplish such processing and how one can reliably benchmark their performance. TRECVID has been funded by NIST (National Institute of Standards and Technology) and other US government agencies. In addition, many organizations and individuals worldwide contribute significant time and effort. TRECVID 2019 represented a continuation of four tasks from TRECVID 2018. In total, 27 teams from various research organizations worldwide completed one or more of the following four tasks: 1. Ad-hoc Video Search (AVS) 2. Instance Search (INS) 3. Activities in Extended Video (ActEV) 4. Video to Text Description (VTT) This paper is an introduction to the evaluation framework, tasks, data, and measures used in the workshop.