Explainable Action Form Assessment by Exploiting Multimodal Chain-of-Thoughts Reasoning
Mengshi Qi, Yeteng Wu, Xianlin Zhang, Huadong Ma

TL;DR
This paper introduces a new task and dataset for explainable human action form assessment, employing a Chain-of-Thought reasoning paradigm to evaluate, explain, and improve action standardization in videos.
Contribution
The paper presents a novel dataset CoT-AFA with multi-level annotations and a framework that provides both action assessment and detailed explanations using multimodal reasoning.
Findings
Improved explanation generation by +16.0% CIDEr score
Enhanced action classification accuracy by +2.7%
Better quality assessment accuracy by +2.1%
Abstract
Evaluating whether human action is standard or not and providing reasonable feedback to improve action standardization is very crucial but challenging in real-world scenarios. However, current video understanding methods are mainly concerned with what and where the action is, which is unable to meet the requirements. Meanwhile, most of the existing datasets lack the labels indicating the degree of action standardization, and the action quality assessment datasets lack explainability and detailed feedback. Therefore, we define a new Human Action Form Assessment (AFA) task, and introduce a new diverse dataset CoT-AFA, which contains a large scale of fitness and martial arts videos with multi-level annotations for comprehensive video analysis. We enrich the CoT-AFA dataset with a novel Chain-of-Thought explanation paradigm. Instead of offering isolated feedback, our explanations provide a…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Pose and Action Recognition · Explainable Artificial Intelligence (XAI) · Generative Adversarial Networks and Image Synthesis
