Hierarchical Video Understanding
Farzaneh Mahdisoltani, Roland Memisevic, David Fleet

TL;DR
This paper presents a hierarchical video understanding model that captures actions at multiple levels of detail, improving performance by leveraging the structure of real-world activities.
Contribution
It introduces a hierarchical architecture trained with a joint loss to learn coarse and fine-grained video targets simultaneously, enhancing understanding accuracy.
Findings
Models exploiting multiple granularity levels outperform single-level models.
Hierarchical training improves accuracy on coarse, fine, and captioning tasks.
Empirical validation on the Something-Something dataset demonstrates effectiveness.
Abstract
We introduce a hierarchical architecture for video understanding that exploits the structure of real world actions by capturing targets at different levels of granularity. We design the model such that it first learns simpler coarse-grained tasks, and then moves on to learn more fine-grained targets. The model is trained with a joint loss on different granularity levels. We demonstrate empirical results on the recent release of Something-Something dataset, which provides a hierarchy of targets, namely coarse-grained action groups, fine-grained action categories, and captions. Experiments suggest that models that exploit targets at different levels of granularity achieve better performance on all levels.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Pose and Action Recognition · Multimodal Machine Learning Applications · Video Analysis and Summarization
