TrUMAn: Trope Understanding in Movies and Animations
Hung-Ting Su, Po-Wei Shen, Bing-Chen Tsai, Wen-Feng Cheng, Ke-Jyun, Wang, Winston H. Hsu

TL;DR
This paper introduces TrUMAn, a new dataset and task for understanding storytelling tropes in videos, aiming to enhance deep cognition in video analysis beyond visual cues.
Contribution
The paper presents a novel trope understanding dataset and a new model, TrUSt, to improve deep reasoning in video comprehension tasks.
Findings
State-of-the-art models achieve only 12-28% accuracy on the task.
TrUSt improves performance to 13.94% accuracy.
Detailed analysis provided to guide future research.
Abstract
Understanding and comprehending video content is crucial for many real-world applications such as search and recommendation systems. While recent progress of deep learning has boosted performance on various tasks using visual cues, deep cognition to reason intentions, motivation, or causality remains challenging. Existing datasets that aim to examine video reasoning capability focus on visual signals such as actions, objects, relations, or could be answered utilizing text bias. Observing this, we propose a novel task, along with a new dataset: Trope Understanding in Movies and Animations (TrUMAn), with 2423 videos associated with 132 tropes, intending to evaluate and develop learning systems beyond visual signals. Tropes are frequently used storytelling devices for creative works. By coping with the trope understanding task and enabling the deep cognition skills of machines, data mining…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMultimodal Machine Learning Applications · Human Pose and Action Recognition · Video Analysis and Summarization
MethodsRefunds@Expedia|||How do I get a full refund from Expedia? · Multi-Head Attention · Attention Is All You Need · Linear Layer · Weight Decay · Softmax · Dense Connections · Dropout · Layer Normalization · WordPiece
