Boundary-Aware Proposal Generation Method for Temporal Action Localization

TL;DR

This paper introduces a boundary-aware proposal generation method using contrastive learning to improve temporal action localization accuracy by better distinguishing action boundaries from background frames.

Contribution

The proposed BAPG method effectively incorporates background frame discrimination via contrastive learning, enhancing existing TAL models without requiring architecture changes.

Findings

Significant performance improvements on THUMOS14 and ActivityNet-1.3 datasets.

Effective background frame discrimination improves boundary detection.

Plug-and-play integration with existing TAL models.

Abstract

The goal of Temporal Action Localization (TAL) is to find the categories and temporal boundaries of actions in an untrimmed video. Most TAL methods rely heavily on action recognition models that are sensitive to action labels rather than temporal boundaries. More importantly, few works consider the background frames that are similar to action frames in pixels but dissimilar in semantics, which also leads to inaccurate temporal boundaries. To address the challenge above, we propose a Boundary-Aware Proposal Generation (BAPG) method with contrastive learning. Specifically, we define the above background frames as hard negative samples. Contrastive learning with hard negative mining is introduced to improve the discrimination of BAPG. BAPG is independent of the existing TAL network architecture, so it can be applied plug-and-play to mainstream TAL models. Extensive experimental results on THUMOS14 and ActivityNet-1.3 demonstrate that BAPG can significantly improve the performance of TAL.