All About Knowledge Graphs for Actions
Pallabi Ghosh, Nirat Saini, Larry S. Davis, Abhinav Shrivastava

TL;DR
This paper investigates how different types of knowledge graphs affect zero-shot and few-shot action recognition, providing insights and an improved evaluation framework for knowledge transfer in video datasets.
Contribution
It systematically analyzes three knowledge graph construction methods for action recognition and proposes a new evaluation paradigm for zero-shot and few-shot learning.
Findings
Action-object embeddings improve recognition accuracy.
Visual embeddings provide complementary benefits.
The proposed evaluation framework standardizes comparisons.
Abstract
Current action recognition systems require large amounts of training data for recognizing an action. Recent works have explored the paradigm of zero-shot and few-shot learning to learn classifiers for unseen categories or categories with few labels. Following similar paradigms in object recognition, these approaches utilize external sources of knowledge (eg. knowledge graphs from language domains). However, unlike objects, it is unclear what is the best knowledge representation for actions. In this paper, we intend to gain a better understanding of knowledge graphs (KGs) that can be utilized for zero-shot and few-shot action recognition. In particular, we study three different construction mechanisms for KGs: action embeddings, action-object embeddings, visual embeddings. We present extensive analysis of the impact of different KGs in different experimental setups. Finally, to enable a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Pose and Action Recognition · Multimodal Machine Learning Applications · Advanced Graph Neural Networks
