SKEL-CF: Coarse-to-Fine Biomechanical Skeleton and Surface Mesh Recovery

TL;DR

SKEL-CF introduces a transformer-based coarse-to-fine framework for accurate, anatomically consistent biomechanical skeleton and surface mesh recovery from images, improving over previous methods in pose estimation accuracy.

Contribution

The paper presents SKEL-CF, a novel transformer-based coarse-to-fine approach for SKEL parameter estimation, with new high-quality training data and explicit camera modeling.

Findings

Achieves 85.0 MPJPE on MOYO dataset, outperforming previous methods.

Demonstrates the importance of camera modeling in diverse viewpoints.

Provides a scalable framework for biomechanical human motion analysis.

Abstract

Parametric 3D human models such as SMPL have driven significant advances in human pose and shape estimation, yet their simplified kinematics limit biomechanical realism. The recently proposed SKEL model addresses this limitation by re-rigging SMPL with an anatomically accurate skeleton. However, estimating SKEL parameters directly remains challenging due to limited training data, perspective ambiguities, and the inherent complexity of human articulation. We introduce SKEL-CF, a coarse-to-fine framework for SKEL parameter estimation. SKEL-CF employs a transformer-based encoder-decoder architecture, where the encoder predicts coarse camera and SKEL parameters, and the decoder progressively refines them in successive layers. To ensure anatomically consistent supervision, we convert the existing SMPL-based dataset 4DHuman into a SKEL-aligned version, 4DHuman-SKEL, providing high-quality training data for SKEL estimation. In addition, to mitigate depth and scale ambiguities, we explicitly incorporate camera modeling into the SKEL-CF pipeline and demonstrate its importance across diverse viewpoints. Extensive experiments validate the effectiveness of the proposed design. On the challenging MOYO dataset, SKEL-CF achieves 85.0 MPJPE / 51.4 PA-MPJPE, significantly outperforming the previous SKEL-based state-of-the-art HSMR (104.5 / 79.6). These results establish SKEL-CF as a scalable and anatomically faithful framework for human motion analysis, facilitating the use of computer vision techniques in biomechanics-related analysis. Our implementation is available on the project page: https://pokerman8.github.io/SKEL-CF/.