An Efficient Federated Distillation Learning System for Multi-task Time Series Classification

TL;DR

This paper introduces an efficient federated distillation learning system for multi-task time series classification, leveraging a feature-based student-teacher framework and a distance-based weights matching scheme to improve accuracy and efficiency.

Contribution

The paper presents a novel federated distillation system with a feature-based student-teacher framework and a distance-based weights matching scheme for multi-task time series classification.

Findings

Achieves high top-1 accuracy on UCR2018 datasets.

Efficient knowledge transfer among users with different TSC tasks.

Effective model weight matching improves federated learning performance.

Abstract

This paper proposes an efficient federated distillation learning system (EFDLS) for multi-task time series classification (TSC). EFDLS consists of a central server and multiple mobile users, where different users may run different TSC tasks. EFDLS has two novel components, namely a feature-based student-teacher (FBST) framework and a distance-based weights matching (DBWM) scheme. Within each user, the FBST framework transfers knowledge from its teacher's hidden layers to its student's hidden layers via knowledge distillation, with the teacher and student having identical network structure. For each connected user, its student model's hidden layers' weights are uploaded to the EFDLS server periodically. The DBWM scheme is deployed on the server, with the least square distance used to measure the similarity between the weights of two given models. This scheme finds a partner for each connected user such that the user's and its partner's weights are the closest among all the weights uploaded. The server exchanges and sends back the user's and its partner's weights to these two users which then load the received weights to their teachers' hidden layers. Experimental results show that the proposed EFDLS achieves excellent performance on a set of selected UCR2018 datasets regarding top-1 accuracy.