Winning the CVPR'2021 Kinetics-GEBD Challenge: Contrastive Learning Approach

TL;DR

This paper presents a contrastive learning approach for generic event boundary detection in videos, leveraging temporal self-similarity matrices to identify natural event boundaries aligned with human perception, achieving top performance in a challenge.

Contribution

Introduces a novel contrastive learning method using temporal self-similarity matrices for GEBD, improving boundary detection accuracy over existing baselines.

Findings

Significant performance boost over baselines.

Effective use of temporal self-similarity matrices.

Achieved top results in CVPR 2021 Kinetics-GEBD Challenge.

Abstract

Generic Event Boundary Detection (GEBD) is a newly introduced task that aims to detect "general" event boundaries that correspond to natural human perception. In this paper, we introduce a novel contrastive learning based approach to deal with the GEBD. Our intuition is that the feature similarity of the video snippet would significantly vary near the event boundaries, while remaining relatively the same in the remaining part of the video. In our model, Temporal Self-similarity Matrix (TSM) is utilized as an intermediate representation which takes on a role as an information bottleneck. With our model, we achieved significant performance boost compared to the given baselines. Our code is available at https://github.com/hello-jinwoo/LOVEU-CVPR2021.