Learning Dynamics from Kinematics: Estimating 2D Foot Pressure Maps from Video Frames

TL;DR

This paper introduces PressNET, a deep learning model that estimates 2D foot pressure maps and CoP locations from video-based human pose data, advancing stability analysis in biomechanics.

Contribution

The paper presents a novel end-to-end CNN architecture for predicting foot pressure heatmaps from pose data, validated on a new Taiji dataset, outperforming baseline methods.

Findings

PressNET significantly outperforms K-Nearest Neighbors baseline.

CoP estimates from PressNET are more accurate than traditional methods.

The approach aligns well with lab-based stability measurements.

Abstract

Pose stability analysis is the key to understanding locomotion and control of body equilibrium, with applications in numerous fields such as kinesiology, medicine, and robotics. In biomechanics, Center of Pressure (CoP) is used in studies of human postural control and gait. We propose and validate a novel approach to learn CoP from pose of a human body to aid stability analysis. More specifically, we propose an end-to-end deep learning architecture to regress foot pressure heatmaps, and hence the CoP locations, from 2D human pose derived from video. We have collected a set of long (5min +) choreographed Taiji (Tai Chi) sequences of multiple subjects with synchronized foot pressure and video data. The derived human pose data and corresponding foot pressure maps are used jointly in training a convolutional neural network with residual architecture, named PressNET. Cross-subject validation results show promising performance of PressNET, significantly outperforming the baseline method of K-Nearest Neighbors. Furthermore, we demonstrate that our computation of center of pressure (CoP) from PressNET is not only significantly more accurate than those obtained from the baseline approach but also meets the expectations of corresponding lab-based measurements of stability studies in kinesiology.