On the Query Strategies for Efficient Online Active Distillation

TL;DR

This paper evaluates query strategies for online active distillation in human pose estimation, demonstrating improved training efficiency and real-time adaptation of lightweight models at the edge.

Contribution

It introduces and assesses query strategies for online active distillation, enabling efficient training and adaptation of lightweight models in real-time HPE applications.

Findings

Effective query strategies improve training efficiency.

Online distillation enables real-time model adaptation.

Lightweight models can be trained at the edge for new contexts.

Abstract

Deep Learning (DL) requires lots of time and data, resulting in high computational demands. Recently, researchers employ Active Learning (AL) and online distillation to enhance training efficiency and real-time model adaptation. This paper evaluates a set of query strategies to achieve the best training results. It focuses on Human Pose Estimation (HPE) applications, assessing the impact of selected frames during training using two approaches: a classical offline method and a online evaluation through a continual learning approach employing knowledge distillation, on a popular state-of-the-art HPE dataset. The paper demonstrates the possibility of enabling training at the edge lightweight models, adapting them effectively to new contexts in real-time.