Deep learning-based estimation of whole-body kinematics from multi-view images

TL;DR

This paper introduces an end-to-end deep learning method for estimating whole-body joint angles directly from multi-view images, utilizing volumetric pose representation and a new dataset, achieving promising accuracy for on-site kinematic analysis.

Contribution

It presents a novel approach for direct joint angle estimation from multi-view images and introduces a new dataset for residential roofing tasks.

Findings

Mean angle error of 7.19° on Roofing dataset

Mean angle error of 8.41° on Human3.6M dataset

Demonstrates feasibility of on-site kinematic analysis

Abstract

It is necessary to analyze the whole-body kinematics (including joint locations and joint angles) to assess risks of fatal and musculoskeletal injuries in occupational tasks. Human pose estimation has gotten more attention in recent years as a method to minimize the errors in determining joint locations. However, the joint angles are not often estimated, nor is the quality of joint angle estimation assessed. In this paper, we presented an end-to-end approach on direct joint angle estimation from multi-view images. Our method leveraged the volumetric pose representation and mapped the rotation representation to a continuous space where each rotation was uniquely represented. We also presented a new kinematic dataset in the domain of residential roofing with a data processing pipeline to generate necessary annotations for the supervised training procedure on direct joint angle estimation. We achieved a mean angle error of $7.19^\circ$ on the new Roofing dataset and $8.41^\circ$ on the Human3.6M dataset, paving the way for employment of on-site kinematic analysis using multi-view images.