Real-Time Cellist Postural Evaluation With On-Device Computer Vision

TL;DR

This paper introduces Cello Evaluator, a real-time, on-device computer vision system that provides posture feedback to cellists using only a modern Android phone, aiming to improve practice quality.

Contribution

The paper presents a mobile app for real-time postural evaluation of cellists, optimized for on-device inference, increasing accessibility and convenience.

Findings

Expert evaluation rated the app as user friendly.

The system successfully runs in real-time on standard Android devices.

The app helps reduce postural issues during practice.

Abstract

Posture is a critical factor for beginning instrumental learners. Most students receive instruction only once a week, and during the intervals between lessons they have little or no feedback on their physical posture. As a result, posture often deteriorates, increasing the risk of musculoskeletal injury and inefficient technique. Recent advances in computer vision and machine learning make it possible to evaluate posture without the constant presence of a human expert. However, current solutions have been extremely limited in availability and convenience due to their reliance on computationally expensive hardware or multi-sensor setups. We present Cello Evaluator, a real-time postural feedback system for practicing cellists. Through this optimization for on-device computer vision inference, we provide access to cellist postural evaluation to anyone with a current generation Android phone and thus reduces the postural feedback voids within individual practice. To validate our mobile application, we conduct a heuristic evaluation consisting of cellist and UX experts. Overall feedback from the evaluation found the app to be user friendly and helpful.