FlexAI: A Multi-modal Solution for Delivering Personalized and Adaptive Fitness Interventions

TL;DR

FlexAI is a multi-modal, AI-driven fitness system that provides real-time, personalized exercise guidance by integrating computer vision, physiological sensors, and large language models, leading to improved user engagement and effectiveness.

Contribution

This work introduces FlexAI, a novel adaptive fitness coaching system that combines multi-modal sensing with LLM reasoning for personalized, real-time workout interventions.

Findings

Participants experienced greater enjoyment and achievement.

FlexAI reduced boredom and frustration during workouts.

System demonstrated high accuracy and low latency in technical evaluations.

Abstract

Personalization of exercise routines is a crucial factor in helping people achieve their fitness goals. Despite this, many contemporary solutions fail to offer real-time, adaptive feedback tailored to an individual's physiological states. Contemporary fitness solutions often rely only on static plans and do not adjust to factors such as a user's pain thresholds, fatigue levels, or form during a workout routine. This work introduces FlexAI, a multi-modal system that integrates computer vision, physiological sensors (heart rate and voice), and the reasoning capabilities of Large Language Models (LLMs) to deliver real-time, personalized workout guidance. FlexAI continuously monitors a user's physical form and level of exertion, among other parameters, to provide dynamic interventions focused on exercise intensity, rest periods, and motivation. To validate our system, we performed a technical evaluation confirming our models' accuracy and quantifying pipeline latency, alongside an expert review where certified trainers validated the correctness of the LLM's interventions. Furthermore, in a controlled study with 25 participants, FlexAI demonstrated significant improvements over a static, non-adaptive control system. With FlexAI, users reported significantly greater enjoyment, a stronger sense of achievement, and significantly lower levels of boredom and frustration. These results indicate that by integrating multi-modal sensing with LLM-driven reasoning, adaptive systems like FlexAI can create a more engaging and effective workout experience. Our work provides a blueprint for integrating multi-modal sensing with LLM-driven reasoning, demonstrating that it is possible to create adaptive coaching systems that are not only more engaging but also demonstrably reliable.