BSN: Boundary Sensitive Network for Temporal Action Proposal Generation

TL;DR

This paper introduces BSN, a boundary-sensitive network that improves temporal action proposal generation by accurately locating boundaries and evaluating proposal confidence, leading to higher recall and precision in long videos.

Contribution

The paper proposes a novel boundary-sensitive network that combines local boundary detection with global proposal evaluation for superior temporal action proposals.

Findings

BSN outperforms existing methods on ActivityNet-1.3 and THUMOS14 datasets.

BSN achieves higher recall and temporal precision in proposal generation.

Combining BSN with classifiers enhances overall action detection performance.

Abstract

Temporal action proposal generation is an important yet challenging problem, since temporal proposals with rich action content are indispensable for analysing real-world videos with long duration and high proportion irrelevant content. This problem requires methods not only generating proposals with precise temporal boundaries, but also retrieving proposals to cover truth action instances with high recall and high overlap using relatively fewer proposals. To address these difficulties, we introduce an effective proposal generation method, named Boundary-Sensitive Network (BSN), which adopts "local to global" fashion. Locally, BSN first locates temporal boundaries with high probabilities, then directly combines these boundaries as proposals. Globally, with Boundary-Sensitive Proposal feature, BSN retrieves proposals by evaluating the confidence of whether a proposal contains an action within its region. We conduct experiments on two challenging datasets: ActivityNet-1.3 and THUMOS14, where BSN outperforms other state-of-the-art temporal action proposal generation methods with high recall and high temporal precision. Finally, further experiments demonstrate that by combining existing action classifiers, our method significantly improves the state-of-the-art temporal action detection performance.