Generalized Uncertainty-Based Evidential Fusion with Hybrid Multi-Head Attention for Weak-Supervised Temporal Action Localization

TL;DR

This paper introduces a novel hybrid multi-head attention and uncertainty-based evidential fusion approach to improve weakly supervised temporal action localization by effectively filtering background noise and enhancing feature representation.

Contribution

It proposes a hybrid multi-head attention module and a generalized uncertainty-based evidential fusion module to better localize and classify actions in videos with weak supervision.

Findings

Outperforms state-of-the-art methods on THUMOS14 dataset

Enhances feature filtering and uncertainty measurement

Improves action localization accuracy

Abstract

Weakly supervised temporal action localization (WS-TAL) is a task of targeting at localizing complete action instances and categorizing them with video-level labels. Action-background ambiguity, primarily caused by background noise resulting from aggregation and intra-action variation, is a significant challenge for existing WS-TAL methods. In this paper, we introduce a hybrid multi-head attention (HMHA) module and generalized uncertainty-based evidential fusion (GUEF) module to address the problem. The proposed HMHA effectively enhances RGB and optical flow features by filtering redundant information and adjusting their feature distribution to better align with the WS-TAL task. Additionally, the proposed GUEF adaptively eliminates the interference of background noise by fusing snippet-level evidences to refine uncertainty measurement and select superior foreground feature information, which enables the model to concentrate on integral action instances to achieve better action localization and classification performance. Experimental results conducted on the THUMOS14 dataset demonstrate that our method outperforms state-of-the-art methods. Our code is available in \url{https://github.com/heyuanpengpku/GUEF/tree/main}.