Cross-Domain Knowledge Distillation for Low-Resolution Human Pose Estimation

TL;DR

This paper introduces a novel cross-domain knowledge distillation framework that improves low-resolution human pose estimation by aligning features across different resolutions and class spaces, validated on MPII and COCO datasets.

Contribution

The work proposes a scale-adaptive projector ensemble and cross-class alignment modules to effectively transfer knowledge from high- to low-resolution models, addressing feature and class mismatch issues.

Findings

Significant performance boost on MPII and COCO datasets.

Effective feature alignment across different input resolutions.

Enhanced distillation with an easy-to-hard training strategy.

Abstract

In practical applications of human pose estimation, low-resolution inputs frequently occur, and existing state-of-the-art models perform poorly with low-resolution images. This work focuses on boosting the performance of low-resolution models by distilling knowledge from a high-resolution model. However, we face the challenge of feature size mismatch and class number mismatch when applying knowledge distillation to networks with different input resolutions. To address this issue, we propose a novel cross-domain knowledge distillation (CDKD) framework. In this framework, we construct a scale-adaptive projector ensemble (SAPE) module to spatially align feature maps between models of varying input resolutions. It adopts a projector ensemble to map low-resolution features into multiple common spaces and adaptively merges them based on multi-scale information to match high-resolution features. Additionally, we construct a cross-class alignment (CCA) module to solve the problem of the mismatch of class numbers. By combining an easy-to-hard training (ETHT) strategy, the CCA module further enhances the distillation performance. The effectiveness and efficiency of our approach are demonstrated by extensive experiments on two common benchmark datasets: MPII and COCO. The code is made available in supplementary material.