The Blessings of Unlabeled Background in Untrimmed Videos
Yuan Liu, Jingyuan Chen, Zhenfang Chen, Bing Deng, Jianqiang Huang,, Hanwang Zhang

TL;DR
This paper introduces a novel deconfounder method that leverages unlabelled background in untrimmed videos to improve weakly-supervised temporal action localization, addressing confounding issues in visual recognition.
Contribution
It proposes a model-agnostic TS-PCA deconfounder that exploits unlabelled background to enhance existing WTAL methods without redesigning models.
Findings
Improves performance of four state-of-the-art WTAL methods.
Achieves better localization accuracy on THUMOS-14 and ActivityNet-1.3 datasets.
Demonstrates the effectiveness of using unlabelled background as a blessing.
Abstract
Weakly-supervised Temporal Action Localization (WTAL) aims to detect the action segments with only video-level action labels in training. The key challenge is how to distinguish the action of interest segments from the background, which is unlabelled even on the video-level. While previous works treat the background as "curses", we consider it as "blessings". Specifically, we first use causal analysis to point out that the common localization errors are due to the unobserved confounder that resides ubiquitously in visual recognition. Then, we propose a Temporal Smoothing PCA-based (TS-PCA) deconfounder, which exploits the unlabelled background to model an observed substitute for the unobserved confounder, to remove the confounding effect. Note that the proposed deconfounder is model-agnostic and non-intrusive, and hence can be applied in any WTAL method without model re-designs. Through…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Pose and Action Recognition · Multimodal Machine Learning Applications · Stroke Rehabilitation and Recovery
