A Review of Deep Learning Techniques for Markerless Human Motion on Synthetic Datasets

TL;DR

This paper reviews deep learning methods for markerless human motion capture using synthetic datasets, highlighting recent advances, challenges, and a model that predicts human skeletons from 2D images.

Contribution

It provides a comprehensive review of deep learning techniques for markerless motion capture and demonstrates a model trained on synthetic datasets to predict human skeletons from 2D images.

Findings

Deep learning improves performance on human motion estimation.

Synthetic datasets enable training models without extensive real-world data.

The proposed model successfully predicts skeletons from 2D images.

Abstract

Markerless motion capture has become an active field of research in computer vision in recent years. Its extensive applications are known in a great variety of fields, including computer animation, human motion analysis, biomedical research, virtual reality, and sports science. Estimating human posture has recently gained increasing attention in the computer vision community, but due to the depth of uncertainty and the lack of the synthetic datasets, it is a challenging task. Various approaches have recently been proposed to solve this problem, many of which are based on deep learning. They are primarily focused on improving the performance of existing benchmarks with significant advances, especially 2D images. Based on powerful deep learning techniques and recently collected real-world datasets, we explored a model that can predict the skeleton of an animation based solely on 2D images. Frames generated from different real-world datasets with synthesized poses using different body shapes from simple to complex. The implementation process uses DeepLabCut on its own dataset to perform many necessary steps, then use the input frames to train the model. The output is an animated skeleton for human movement. The composite dataset and other results are the "ground truth" of the deep model.