Look One and More: Distilling Hybrid Order Relational Knowledge for Cross-Resolution Image Recognition

TL;DR

This paper introduces a hybrid order relational knowledge distillation method using a teacher-student framework to improve low-resolution image recognition by transferring high-resolution knowledge, enhancing accuracy across multiple tasks.

Contribution

It proposes a novel hybrid order relational knowledge distillation approach with an auxiliary stream to effectively transfer high-resolution recognition capabilities to low-resolution models.

Findings

Improves low-resolution image recognition accuracy

Effective knowledge transfer across different image recognition tasks

Reduces model complexity while maintaining performance

Abstract

In spite of great success in many image recognition tasks achieved by recent deep models, directly applying them to recognize low-resolution images may suffer from low accuracy due to the missing of informative details during resolution degradation. However, these images are still recognizable for subjects who are familiar with the corresponding high-resolution ones. Inspired by that, we propose a teacher-student learning approach to facilitate low-resolution image recognition via hybrid order relational knowledge distillation. The approach refers to three streams: the teacher stream is pretrained to recognize high-resolution images in high accuracy, the student stream is learned to identify low-resolution images by mimicking the teacher's behaviors, and the extra assistant stream is introduced as bridge to help knowledge transfer across the teacher to the student. To extract sufficient knowledge for reducing the loss in accuracy, the learning of student is supervised with multiple losses, which preserves the similarities in various order relational structures. In this way, the capability of recovering missing details of familiar low-resolution images can be effectively enhanced, leading to a better knowledge transfer. Extensive experiments on metric learning, low-resolution image classification and low-resolution face recognition tasks show the effectiveness of our approach, while taking reduced models.