Part Segmentation of Human Meshes via Multi-View Human Parsing

TL;DR

This paper introduces a geometric method for semantic segmentation of human meshes using multi-view parsing and point cloud deep learning, achieving accurate per-vertex labeling without texture data.

Contribution

It develops a novel pseudo-ground truth labeling pipeline and a memory-efficient sampling strategy for effective human mesh segmentation.

Findings

High accuracy in per-vertex human mesh segmentation.

Effective pseudo-labeling pipeline for large datasets.

Memory-efficient sampling improves computational performance.

Abstract

Recent advances in point cloud deep learning have led to models that achieve high per-part labeling accuracy on large-scale point clouds, using only the raw geometry of unordered point sets. In parallel, the field of human parsing focuses on predicting body part and clothing/accessory labels from images. This work aims to bridge these two domains by enabling per-vertex semantic segmentation of large-scale human meshes. To achieve this, a pseudo-ground truth labeling pipeline is developed for the Thuman2.1 dataset: meshes are first aligned to a canonical pose, segmented from multiple viewpoints, and the resulting point-level labels are then backprojected onto the original mesh to produce per-point pseudo ground truth annotations. Subsequently, a novel, memory-efficient sampling strategy is introduced, a windowed iterative farthest point sampling (FPS) with space-filling curve-based serialization to effectively downsample the point clouds. This is followed by a purely geometric segmentation using PointTransformer, enabling semantic parsing of human meshes without relying on texture information. Experimental results confirm the effectiveness and accuracy of the proposed approach. Project code and pre-processed data is available at https://github.com/JamesMcCullochDickens/Human3DParsing/tree/master.