Chatting about Upper-Body Expressive Human Pose and Shape Estimation

TL;DR

This paper introduces CoEvoer, a transformer-based framework for upper-body expressive human pose and shape estimation, improving accuracy and generalization by modeling interactions among face, hands, and torso.

Contribution

The paper presents CoEvoer, the first framework specifically designed for upper-body EHPS that leverages cross-dependency transformers for joint parameter estimation.

Findings

Achieves state-of-the-art performance on upper-body benchmarks.

Demonstrates strong generalization to unseen wild images.

Enables explicit feature interaction across body parts for improved accuracy.

Abstract

Expressive Human Pose and Shape Estimation (EHPS) plays a crucial role in various AR/VR applications and has witnessed significant progress in recent years. However, current state-of-the-art methods still struggle with accurate parameter estimation for facial and hand regions and exhibit limited generalization to wild images. To address these challenges, we present CoEvoer, a novel one-stage synergistic cross-dependency transformer framework tailored for upper-body EHPS. CoEvoer enables explicit feature-level interaction across different body parts, allowing for mutual enhancement through contextual information exchange. Specifically, larger and more easily estimated regions such as the torso provide global semantics and positional priors to guide the estimation of finer, more complex regions like the face and hands. Conversely, the localized details captured in facial and hand regions help refine and calibrate adjacent body parts. To the best of our knowledge, CoEvoer is the first framework designed specifically for upper-body EHPS, with the goal of capturing the strong coupling and semantic dependencies among the face, hands, and torso through joint parameter regression. Extensive experiments demonstrate that CoEvoer achieves state-of-the-art performance on upper-body benchmarks and exhibits strong generalization capability even on unseen wild images.