Fine-grained Dynamic Network for Generic Event Boundary Detection

TL;DR

This paper introduces DyBDet, a dynamic, multi-exit network architecture that adaptively detects generic event boundaries in videos, improving accuracy and efficiency over previous methods.

Contribution

The paper presents a novel dynamic pipeline with a multi-exit network and multi-order difference detector for fine-grained, adaptive event boundary detection.

Findings

Significant performance improvements on Kinetics-GEBD and TAPOS datasets.

Enhanced detection accuracy and efficiency compared to state-of-the-art methods.

Effective handling of diverse boundary characteristics through adaptive detection.

Abstract

Generic event boundary detection (GEBD) aims at pinpointing event boundaries naturally perceived by humans, playing a crucial role in understanding long-form videos. Given the diverse nature of generic boundaries, spanning different video appearances, objects, and actions, this task remains challenging. Existing methods usually detect various boundaries by the same protocol, regardless of their distinctive characteristics and detection difficulties, resulting in suboptimal performance. Intuitively, a more intelligent and reasonable way is to adaptively detect boundaries by considering their special properties. In light of this, we propose a novel dynamic pipeline for generic event boundaries named DyBDet. By introducing a multi-exit network architecture, DyBDet automatically learns the subnet allocation to different video snippets, enabling fine-grained detection for various boundaries. Besides, a multi-order difference detector is also proposed to ensure generic boundaries can be effectively identified and adaptively processed. Extensive experiments on the challenging Kinetics-GEBD and TAPOS datasets demonstrate that adopting the dynamic strategy significantly benefits GEBD tasks, leading to obvious improvements in both performance and efficiency compared to the current state-of-the-art.