TechCoach: Towards Technical-Point-Aware Descriptive Action Coaching
Yuan-Ming Li, An-Lan Wang, Kun-Yu Lin, Yu-Ming Tang, Ling-An Zeng,, Jian-Fang Hu, Wei-Shi Zheng

TL;DR
This paper introduces TechCoach, a framework for detailed, point-aware action coaching that provides comprehensive feedback on action quality at the TechPoint level, supported by a new dataset and benchmark.
Contribution
It presents a novel dataset EE4D-DescCoach with detailed TechPoint annotations and a framework that explicitly reasons about TechPoints for improved descriptive coaching.
Findings
TechCoach outperforms baseline models in providing detailed coaching comments.
The dataset enables fine-grained action assessment at the TechPoint level.
Explicit TechPoint reasoning improves the quality of coaching feedback.
Abstract
To guide a learner in mastering action skills, it is crucial for a coach to 1) reason through the learner's action execution and technical points (TechPoints), and 2) provide detailed, comprehensible feedback on what is done well and what can be improved. However, existing score-based action assessment methods are still far from reaching this practical scenario. To bridge this gap, we investigate a new task termed Descriptive Action Coaching (DescCoach) which requires the model to provide detailed commentary on what is done well and what can be improved beyond a simple quality score for action execution. To this end, we first build a new dataset named EE4D-DescCoach. Through an automatic annotation pipeline, our dataset goes beyond the existing action assessment datasets by providing detailed TechPoint-level commentary. Furthermore, we propose TechCoach, a new framework that explicitly…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
Taxonomy
TopicsEducational Games and Gamification · Intelligent Tutoring Systems and Adaptive Learning
MethodsDynamic Algorithm Configuration
