Structured Context Learning for Generic Event Boundary Detection

TL;DR

This paper introduces Structured Context Learning with SPoS for efficient and accurate generic event boundary detection in videos, outperforming existing methods across multiple datasets.

Contribution

It proposes a novel Structured Partition of Sequence (SPoS) method that provides structured context for temporal learning, improving speed and accuracy in event boundary detection.

Findings

Achieves better speed-accuracy trade-off than prior methods

Demonstrates superior performance on Kinetics-GEBD, TAPOS, and shot transition datasets

SPoS's complexity is linear with video length

Abstract

Generic Event Boundary Detection (GEBD) aims to identify moments in videos that humans perceive as event boundaries. This paper proposes a novel method for addressing this task, called Structured Context Learning, which introduces the Structured Partition of Sequence (SPoS) to provide a structured context for learning temporal information. Our approach is end-to-end trainable and flexible, not restricted to specific temporal models like GRU, LSTM, and Transformers. This flexibility enables our method to achieve a better speed-accuracy trade-off. Specifically, we apply SPoS to partition the input frame sequence and provide a structured context for the subsequent temporal model. Notably, SPoS's overall computational complexity is linear with respect to the video length. We next calculate group similarities to capture differences between frames, and a lightweight fully convolutional network is utilized to determine the event boundaries based on the grouped similarity maps. To remedy the ambiguities of boundary annotations, we adapt the Gaussian kernel to preprocess the ground-truth event boundaries. Our proposed method has been extensively evaluated on the challenging Kinetics-GEBD, TAPOS, and shot transition detection datasets, demonstrating its superiority over existing state-of-the-art methods.