FormCoach: Lift Smarter, Not Harder

TL;DR

FormCoach is an AI-powered system that uses vision-language models to provide real-time, personalized form correction for at-home fitness, addressing the lack of expert feedback for users.

Contribution

This work introduces a novel vision-language model approach for real-time exercise form correction and provides a benchmark dataset and evaluation pipeline for future research.

Findings

Benchmark results show significant gaps compared to human coaching.

The dataset includes 1,700 expert-annotated videos across 22 exercises.

FormCoach demonstrates potential but still faces challenges in nuanced movement analysis.

Abstract

Good form is the difference between strength and strain, yet for the fast-growing community of at-home fitness enthusiasts, expert feedback is often out of reach. FormCoach transforms a simple camera into an always-on, interactive AI training partner, capable of spotting subtle form errors and delivering tailored corrections in real time, leveraging vision-language models (VLMs). We showcase this capability through a web interface and benchmark state-of-the-art VLMs on a dataset of 1,700 expert-annotated user-reference video pairs spanning 22 strength and mobility exercises. To accelerate research in AI-driven coaching, we release both the dataset and an automated, rubric-based evaluation pipeline, enabling standardized comparison across models. Our benchmarks reveal substantial gaps compared to human-level coaching, underscoring both the challenges and opportunities in integrating nuanced, context-aware movement analysis into interactive AI systems. By framing form correction as a collaborative and creative process between humans and machines, FormCoach opens a new frontier in embodied AI.