ProTAL: A Drag-and-Link Video Programming Framework for Temporal Action Localization
Yuchen He, Jianbing Lv, Liqi Cheng, Lingyu Meng, Dazhen Deng, Yingcai Wu

TL;DR
ProTAL introduces a user-friendly drag-and-link framework for defining key events in videos, enabling efficient label generation for temporal action localization and reducing manual annotation efforts.
Contribution
It presents a novel video programming framework that simplifies defining complex actions and generates training labels for TAL using semi-supervised learning.
Findings
Effective label generation for unlabelled videos
User study shows ease of defining key events
Improves TAL model training with less manual annotation
Abstract
Temporal Action Localization (TAL) aims to detect the start and end timestamps of actions in a video. However, the training of TAL models requires a substantial amount of manually annotated data. Data programming is an efficient method to create training labels with a series of human-defined labeling functions. However, its application in TAL faces difficulties of defining complex actions in the context of temporal video frames. In this paper, we propose ProTAL, a drag-and-link video programming framework for TAL. ProTAL enables users to define \textbf{key events} by dragging nodes representing body parts and objects and linking them to constrain the relations (direction, distance, etc.). These definitions are used to generate action labels for large-scale unlabelled videos. A semi-supervised method is then employed to train TAL models with such labels. We demonstrate the effectiveness…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
