Cascaded Boundary Regression for Temporal Action Detection

TL;DR

This paper introduces Cascaded Boundary Regression, a two-stage approach for temporal action detection that refines action boundaries through cascaded coordinate regression, achieving state-of-the-art results on benchmark datasets.

Contribution

The paper proposes a novel cascaded boundary refinement method within a two-stage detection pipeline for improved temporal action localization.

Findings

Achieves state-of-the-art performance on THUMOS-14 and TVSeries datasets.

Significant improvement at high IoU thresholds, e.g., mAP@0.5 on THUMOS-14 from 19.0% to 31.0%.

Effective boundary refinement through cascaded regression enhances detection accuracy.

Abstract

Temporal action detection in long videos is an important problem. State-of-the-art methods address this problem by applying action classifiers on sliding windows. Although sliding windows may contain an identifiable portion of the actions, they may not necessarily cover the entire action instance, which would lead to inferior performance. We adapt a two-stage temporal action detection pipeline with Cascaded Boundary Regression (CBR) model. Class-agnostic proposals and specific actions are detected respectively in the first and the second stage. CBR uses temporal coordinate regression to refine the temporal boundaries of the sliding windows. The salient aspect of the refinement process is that, inside each stage, the temporal boundaries are adjusted in a cascaded way by feeding the refined windows back to the system for further boundary refinement. We test CBR on THUMOS-14 and TVSeries, and achieve state-of-the-art performance on both datasets. The performance gain is especially remarkable under high IoU thresholds, e.g. map@tIoU=0.5 on THUMOS-14 is improved from 19.0% to 31.0%.