KITRO: Refining Human Mesh by 2D Clues and Kinematic-tree Rotation

TL;DR

KITRO introduces a novel mesh refinement method that explicitly models depth and kinematic-tree structure, improving 3D human pose estimation accuracy by calculating bone directions in closed form and using a decision tree for hypothesis selection.

Contribution

The paper presents KITRO, a new approach that refines 3D human meshes by explicitly modeling depth and kinematic structure, moving beyond gradient-based optimization.

Findings

Significantly improves 3D joint estimation accuracy.

Achieves better 2D fit while refining meshes.

Demonstrates effectiveness across various datasets and models.

Abstract

2D keypoints are commonly used as an additional cue to refine estimated 3D human meshes. Current methods optimize the pose and shape parameters with a reprojection loss on the provided 2D keypoints. Such an approach, while simple and intuitive, has limited effectiveness because the optimal solution is hard to find in ambiguous parameter space and may sacrifice depth. Additionally, divergent gradients from distal joints complicate and deviate the refinement of proximal joints in the kinematic chain. To address these, we introduce Kinematic-Tree Rotation (KITRO), a novel mesh refinement strategy that explicitly models depth and human kinematic-tree structure. KITRO treats refinement from a bone-wise perspective. Unlike previous methods which perform gradient-based optimizations, our method calculates bone directions in closed form. By accounting for the 2D pose, bone length, and parent joint's depth, the calculation results in two possible directions for each child joint. We then use a decision tree to trace binary choices for all bones along the human skeleton's kinematic-tree to select the most probable hypothesis. Our experiments across various datasets and baseline models demonstrate that KITRO significantly improves 3D joint estimation accuracy and achieves an ideal 2D fit simultaneously. Our code available at: https://github.com/MartaYang/KITRO.