Modelling the Distribution of Human Motion for Sign Language Assessment
Oliver Cory, Ozge Mercanoglu Sincan, Matthew Vowels, Alessia Battisti,, Franz Holzknecht, Katja Tissi, Sandra Sidler-Miserez, Tobias Haug, Sarah, Ebling, Richard Bowden
TL;DR
This paper presents a new sign language assessment tool that models natural human motion distribution to evaluate comprehensibility, showing strong correlation with human ratings and capable of detecting anomalies for improved learning feedback.
Contribution
Introduces a novel SLA tool that models natural human motion distribution for evaluating sign language comprehensibility, advancing beyond isolated sign analysis.
Findings
Strong correlation with human ratings
Effective detection of anomalous sign motions
Provides actionable feedback for sign language learning
Abstract
Sign Language Assessment (SLA) tools are useful to aid in language learning and are underdeveloped. Previous work has focused on isolated signs or comparison against a single reference video to assess Sign Languages (SL). This paper introduces a novel SLA tool designed to evaluate the comprehensibility of SL by modelling the natural distribution of human motion. We train our pipeline on data from native signers and evaluate it using SL learners. We compare our results to ratings from a human raters study and find strong correlation between human ratings and our tool. We visually demonstrate our tools ability to detect anomalous results spatio-temporally, providing actionable feedback to aid in SL learning and assessment.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHand Gesture Recognition Systems · Gait Recognition and Analysis · Hearing Impairment and Communication