Real-Time System for Human Activity Analysis

TL;DR

This paper introduces a real-time human activity analysis system using dual Kinects, SVD, SQP, and IDTW to evaluate and visually feedback user performance, improving learning outcomes.

Contribution

The novel system combines dual Kinect sensors, advanced joint extraction, and real-time scoring with visual feedback to enhance activity analysis and learning.

Findings

Dual Kinect setup improves joint position accuracy

Visual feedback significantly boosts user learning

System achieves real-time performance with high accuracy

Abstract

We propose a real-time human activity analysis system, where a user's activity can be quantiatively evaluated with respect to a ground truth recording. We use two Kinects to solve the ptorblem of self-occlusion through extraction optimal joint positions using Singular Value Decomposition (SVD) and Sequential Quadratic Programming (SQP). Incremental Dynamic Time Warping (IDTW) is used to compare the user and expert (ground truth) to quantiatively score the user's performance. Furthermore, the user's performance is displayed through a visual feedback system, where colors on the skeleton represent the user's score. Our experiements use a motion capture suit as ground truth to compare our dual Kinect setup to a single Kinect. We also show that with out visual feedback method, users gain statistically significant boost to learning as opposed to watching a simple video.