Development of user-customized online teaching technology based on GPT, enhanced by generative AI-based motion recognition

TL;DR

This paper introduces ND_GAN, a new motion recognition system that improves accuracy in complex environments and integrates GPT for real-time feedback in online teaching.

Contribution

The novel ND_GAN model enhances joint tracking accuracy by over 30% compared to existing models like MediaPipe and provides real-time teaching feedback using GPT.

Findings

ND_GAN achieved 94.2% PCK@0.5 and 93.1% PCK@0.2, surpassing MediaPipe by over 30%.

ND_GAN outperformed DWPose and ViTPose by more than 15% in accuracy under occlusion conditions.

The system provides real-time feedback using GPT, improving user learning effectiveness.

Abstract

This paper addresses current issues in existing joint tracking and motion recognition algorithms for Human Pose Estimation. It proposes a solution using the Numerical Discriminator Generative Adversarial Network (ND_GAN) to improve the performance of vision-based motion recognition technology. Existing algorithms face challenges in accurately tracking joints in crowded spaces or with users wearing special attire, resulting in reduced accuracy and inconsistent results. The proposed ND_GAN consists of three integrated modules, enabling more precise joint estimation even in complex environments. Experiments were conducted using yoga videos from Hanchoom, a home training platform by TDI, comparing the performance of MediaPipe and ND_GAN on both original and leg-masked videos. Quantitative evaluation showed that ND_GAN achieved 94.2% PCK@0.5, 93.1% PCK@0.2, and an F1-score of 92.8%, marking an improvement of over 30% compared to the existing MediaPipe model. Notably, ND_GAN consistently estimated joint coordinates even in occluded video conditions, demonstrating significant performance gains in scenarios where traditional models failed. Furthermore, ND_GAN outperformed cutting-edge state-of-the-art models such as DWPose and ViTPose by more than 15% in accuracy, confirming its robustness in real-world environments with various occlusion conditions. Additionally, a teaching system using GPT is introduced to provide real-time feedback on movement errors, supporting users in effectively learning the motions. This integrated approach marks a significant advancement in addressing Human Pose Estimation complexities, enhancing overall vision-based motion recognition efficacy.

Figures