A Neural Network Approach to Missing Marker Reconstruction in Human Motion Capture

TL;DR

This paper presents neural network models, including LSTM and time-window approaches, that accurately reconstruct missing markers in human motion capture data in real-time, outperforming existing methods.

Contribution

It introduces two novel neural network models capable of online reconstruction of missing markers in motion capture, leveraging temporal and spatial correlations.

Findings

Both models achieve state-of-the-art accuracy.

Models operate online without needing complete sequence data.

Implementation is publicly available.

Abstract

Optical motion capture systems have become a widely used technology in various fields, such as augmented reality, robotics, movie production, etc. Such systems use a large number of cameras to triangulate the position of optical markers.The marker positions are estimated with high accuracy. However, especially when tracking articulated bodies, a fraction of the markers in each timestep is missing from the reconstruction. In this paper, we propose to use a neural network approach to learn how human motion is temporally and spatially correlated, and reconstruct missing markers positions through this model. We experiment with two different models, one LSTM-based and one time-window-based. Both methods produce state-of-the-art results, while working online, as opposed to most of the alternative methods, which require the complete sequence to be known. The implementation is publicly available at https://github.com/Svito-zar/NN-for-Missing-Marker-Reconstruction .