Towards Adaptive Pseudo-label Learning for Semi-Supervised Temporal Action Localization

TL;DR

This paper introduces an Adaptive Pseudo-label Learning framework for semi-supervised temporal action localization, improving pseudo-label quality through joint assessment and instance-level discrimination, leading to state-of-the-art results.

Contribution

The paper proposes a novel adaptive pseudo-label selection method combining classification and localization assessment with an instance-level discriminator and contrastive pre-training.

Findings

Achieves state-of-the-art performance on THUMOS14 and ActivityNet v1.3

Effectively filters pseudo labels with joint quality assessment

Enhances discrimination with unsupervised contrastive pre-training

Abstract

Alleviating noisy pseudo labels remains a key challenge in Semi-Supervised Temporal Action Localization (SS-TAL). Existing methods often filter pseudo labels based on strict conditions, but they typically assess classification and localization quality separately, leading to suboptimal pseudo-label ranking and selection. In particular, there might be inaccurate pseudo labels within selected positives, alongside reliable counterparts erroneously assigned to negatives. To tackle these problems, we propose a novel Adaptive Pseudo-label Learning (APL) framework to facilitate better pseudo-label selection. Specifically, to improve the ranking quality, Adaptive Label Quality Assessment (ALQA) is proposed to jointly learn classification confidence and localization reliability, followed by dynamically selecting pseudo labels based on the joint score. Additionally, we propose an Instance-level Consistency Discriminator (ICD) for eliminating ambiguous positives and mining potential positives simultaneously based on inter-instance intrinsic consistency, thereby leading to a more precise selection. We further introduce a general unsupervised Action-aware Contrastive Pre-training (ACP) to enhance the discrimination both within actions and between actions and backgrounds, which benefits SS-TAL. Extensive experiments on THUMOS14 and ActivityNet v1.3 demonstrate that our method achieves state-of-the-art performance under various semi-supervised settings.