TaiChi Action Capture and Performance Analysis with Multi-view RGB Cameras

TL;DR

This paper presents a novel multi-view RGB camera system and AI-based framework for capturing and analyzing TaiChi movements, enabling detailed 3D reconstruction and performance assessment without markers.

Contribution

It introduces a new multi-camera setup, combines traditional visual methods with neural radiance fields, and normalizes TaiChi movements for performance analysis across groups.

Findings

Effective 3D skeleton and surface reconstruction achieved

Framework demonstrates high efficiency in experiments

Enables performance analysis for different groups

Abstract

Recent advances in computer vision and deep learning have influenced the field of sports performance analysis for researchers to track and reconstruct freely moving humans without any marker attachment. However, there are few works for vision-based motion capture and intelligent analysis for professional TaiChi movement. In this paper, we propose a framework for TaiChi performance capture and analysis with multi-view geometry and artificial intelligence technology. The main innovative work is as follows: 1) A multi-camera system suitable for TaiChi motion capture is built and the multi-view TaiChi data is collected and processed; 2) A combination of traditional visual method and implicit neural radiance field is proposed to achieve sparse 3D skeleton fusion and dense 3D surface reconstruction. 3) The normalization modeling of movement sequences is carried out based on motion transfer, so as to realize TaiChi performance analysis for different groups. We have carried out evaluation experiments, and the experimental results have shown the efficiency of our method.