Mitigating Resolution-Drift in Federated Learning: Case of Keypoint Detection

TL;DR

This paper introduces resolution-adaptive federated learning (RAF), a novel method to mitigate resolution drift in distributed human pose estimation, improving robustness and performance across clients with varying image resolutions.

Contribution

We propose RAF, a heatmap-based knowledge distillation approach that addresses resolution variability in federated learning for non-classification tasks, a previously underexplored area.

Findings

RAF effectively mitigates resolution drift.

Significant performance improvements in human pose estimation.

The method is adaptable to other spatially sensitive tasks.

Abstract

The Federated Learning (FL) approach enables effective learning across distributed systems, while preserving user data privacy. To date, research has primarily focused on addressing statistical heterogeneity and communication efficiency, through which FL has achieved success in classification tasks. However, its application to non-classification tasks, such as human pose estimation, remains underexplored. This paper identifies and investigates a critical issue termed ``resolution-drift,'' where performance degrades significantly due to resolution variability across clients. Unlike class-level heterogeneity, resolution drift highlights the importance of resolution as another axis of not independent or identically distributed (non-IID) data. To address this issue, we present resolution-adaptive federated learning (RAF), a method that leverages heatmap-based knowledge distillation. Through multi-resolution knowledge distillation between higher-resolution outputs (teachers) and lower-resolution outputs (students), our approach enhances resolution robustness without overfitting. Extensive experiments and theoretical analysis demonstrate that RAF not only effectively mitigates resolution drift and achieves significant performance improvements, but also can be integrated seamlessly into existing FL frameworks. Furthermore, although this paper focuses on human pose estimation, our t-SNE analysis reveals distinct characteristics between classification and high-resolution representation tasks, supporting the generalizability of RAF to other tasks that rely on preserving spatial detail.