# Federated Learning for Human Pose Estimation on Non-IID Data via Gradient Coordination

## TL;DR

This paper introduces a new federated learning method to improve human pose estimation accuracy when data is unevenly distributed across devices.

## Contribution

FedGH is a novel aggregation strategy that reduces gradient conflicts in non-IID federated learning for pose estimation.

## Key findings

- FedGH outperforms FedAW by 1.82 and 0.36 percentage points in PCK on two datasets.
- FedGH achieves 86.4% PCK for shoulder detection, surpassing other methods by 20–30%.
- The method reaches over 98% accuracy in keypoint heatmap regression within 10 rounds.

## Abstract

Human pose estimation is an important downstream task in computer vision, with significant applications in action recognition and virtual reality. However, data collected in a decentralized manner often exhibit non-independent and identically distributed (non-IID) characteristics, and traditional federated learning aggregation strategies can lead to gradient conflicts that impair model convergence and accuracy. To address this, we propose the Federated Gradient Harmonization aggregation strategy (FedGH), which coordinates update directions by measuring client gradient discrepancies and integrating gradient-projection correction with a parameter-reconstruction mechanism. Experiments conducted on a self-constructed single-arm robotic dataset and the public Max Planck Institute for Informatics (MPII Human Pose Dataset) dataset demonstrate that FedGH achieves average Percentage of Correct Keypoints (PCK) of 47.14% and 66.31% across all keypoints, representing improvements of 1.82 and 0.36 percentage points over the Federated Adaptive Weighting (FedAW) method. On our self-constructed dataset, FedGH attains a PCK of 86.4% for shoulder detection, surpassing other traditional federated learning methods by 20–30%. Moreover, on the self-constructed dataset, FedGH reaches over 98% accuracy in the keypoint heatmap regression model within the first 10 rounds and remains stable between 98% and 100% thereafter. This method effectively mitigates gradient conflicts in non-IID environments, providing a more robust optimization solution for distributed human pose estimation.