Masked Autoencoders for Generic Event Boundary Detection CVPR'2022 Kinetics-GEBD Challenge

TL;DR

This paper introduces a novel approach using Masked Autoencoders combined with semi-supervised learning and boundary refinement techniques to improve generic event boundary detection in videos, achieving state-of-the-art results.

Contribution

The paper presents a new ensemble of Masked Autoencoders with semi-supervised pseudo-labeling and boundary refinement for GEBD, significantly enhancing detection accuracy.

Findings

Achieved 85.94% F1-score on Kinetics-GEBD test set.

Improved F1-score by 2.31% over previous best.

Demonstrated effectiveness of semi-supervised and boundary refinement methods.

Abstract

Generic Event Boundary Detection (GEBD) tasks aim at detecting generic, taxonomy-free event boundaries that segment a whole video into chunks. In this paper, we apply Masked Autoencoders to improve algorithm performance on the GEBD tasks. Our approach mainly adopted the ensemble of Masked Autoencoders fine-tuned on the GEBD task as a self-supervised learner with other base models. Moreover, we also use a semi-supervised pseudo-label method to take full advantage of the abundant unlabeled Kinetics-400 data while training. In addition, we propose a soft-label method to partially balance the positive and negative samples and alleviate the problem of ambiguous labeling in this task. Lastly, a tricky segmentation alignment policy is implemented to refine boundaries predicted by our models to more accurate locations. With our approach, we achieved 85.94% on the F1-score on the Kinetics-GEBD test set, which improved the F1-score by 2.31% compared to the winner of the 2021 Kinetics-GEBD Challenge. Our code is available at https://github.com/ContentAndMaterialPortrait/MAE-GEBD.