Federated Action Recognition on Heterogeneous Embedded Devices

TL;DR

This paper presents a federated learning approach for action recognition on heterogeneous embedded devices, combining model compression, asynchronous optimization, and empirical validation to achieve efficient training with comparable accuracy.

Contribution

It introduces an asynchronous federated learning method with model compression and fine-tuning for action recognition on resource-limited devices, demonstrating reduced training time.

Findings

Achieves comparable accuracy to baseline methods.

Reduces training time by 40% with asynchronous learning.

Validates approach on heterogeneous embedded devices.

Abstract

Federated learning allows a large number of devices to jointly learn a model without sharing data. In this work, we enable clients with limited computing power to perform action recognition, a computationally heavy task. We first perform model compression at the central server through knowledge distillation on a large dataset. This allows the model to learn complex features and serves as an initialization for model fine-tuning. The fine-tuning is required because the limited data present in smaller datasets is not adequate for action recognition models to learn complex spatio-temporal features. Because the clients present are often heterogeneous in their computing resources, we use an asynchronous federated optimization and we further show a convergence bound. We compare our approach to two baseline approaches: fine-tuning at the central server (no clients) and fine-tuning using (heterogeneous) clients using synchronous federated averaging. We empirically show on a testbed of heterogeneous embedded devices that we can perform action recognition with comparable accuracy to the two baselines above, while our asynchronous learning strategy reduces the training time by 40%, relative to synchronous learning.