Expanding mmWave Datasets for Human Pose Estimation with Unlabeled Data and LiDAR Datasets

TL;DR

This paper introduces EMDUL, a method to augment mmWave human pose datasets using unlabeled mmWave data and LiDAR datasets, improving model accuracy and generalization.

Contribution

EMDUL is a novel approach that combines pseudo-labeling and LiDAR-to-mmWave conversion to expand dataset diversity and size.

Findings

Significant reduction in pose estimation error: 15.1% in-domain and 18.9% out-of-domain.

Enhanced model generalization due to increased dataset diversity.

Effective use of unlabeled data and LiDAR datasets to improve mmWave HPE performance.

Abstract

Current millimeter-wave (mmWave) datasets for human pose estimation (HPE) are scarce and lack diversity in both point cloud (PC) attributes and human poses, hindering the generalization ability of their trained models. On the other hand, unlabeled mmWave HPE data and diverse LiDAR HPE datasets are readily available. We propose EMDUL, a novel approach to expand the volume and diversity of an existing mmWave dataset using unlabeled mmWave data and LiDAR datasets. EMDUL consists of two independent modules, namely a pseudo-label estimator to annotate unlabeled mmWave data, and a closed-form converter that translates an annotated LiDAR PC to its mmWave counterpart. Expanding the original dataset with both LiDAR-converted and pseudo-labeled mmWave PCs significantly boosts the performance and generalization ability of all the examined HPE models, reducing 15.1% and 18.9% error for in-domain and out-of-domain settings, respectively. Code is available at https://github.com/Shimmer93/EMDUL.