A Large-scale Comprehensive Dataset and Copy-overlap Aware Evaluation Protocol for Segment-level Video Copy Detection

TL;DR

This paper introduces VCSL, a large-scale, detailed dataset for segment-level video copy detection, along with a new evaluation protocol, enabling more accurate assessment of video copy detection methods.

Contribution

The paper presents VCSL, a comprehensive dataset with extensive segment-level annotations and a novel evaluation protocol for improved measurement of video copy detection accuracy.

Findings

Benchmarking reveals strengths and weaknesses of current methods.

The new dataset and protocol facilitate better evaluation.

Baseline methods show room for improvement.

Abstract

In this paper, we introduce VCSL (Video Copy Segment Localization), a new comprehensive segment-level annotated video copy dataset. Compared with existing copy detection datasets restricted by either video-level annotation or small-scale, VCSL not only has two orders of magnitude more segment-level labelled data, with 160k realistic video copy pairs containing more than 280k localized copied segment pairs, but also covers a variety of video categories and a wide range of video duration. All the copied segments inside each collected video pair are manually extracted and accompanied by precisely annotated starting and ending timestamps. Alongside the dataset, we also propose a novel evaluation protocol that better measures the prediction accuracy of copy overlapping segments between a video pair and shows improved adaptability in different scenarios. By benchmarking several baseline and state-of-the-art segment-level video copy detection methods with the proposed dataset and evaluation metric, we provide a comprehensive analysis that uncovers the strengths and weaknesses of current approaches, hoping to open up promising directions for future works. The VCSL dataset, metric and benchmark codes are all publicly available at https://github.com/alipay/VCSL.