Federated Learning for Collaborative Inference Systems: The Case of Early Exit Networks

TL;DR

This paper introduces a federated learning approach tailored for cooperative inference systems with hierarchical models, addressing client heterogeneity and improving training efficiency for IoT devices with limited resources.

Contribution

We propose a novel federated learning framework specifically designed for hierarchical cooperative inference systems, accounting for client heterogeneity and providing theoretical guarantees.

Findings

Outperforms state-of-the-art algorithms in heterogeneous scenarios

Provides theoretical guarantees for federated training in CISs

Enhances inference efficiency for IoT devices with limited resources

Abstract

As Internet of Things (IoT) technology advances, end devices like sensors and smartphones are progressively equipped with AI models tailored to their local memory and computational constraints. Local inference reduces communication costs and latency; however, these smaller models typically underperform compared to more sophisticated models deployed on edge servers or in the cloud. Cooperative Inference Systems (CISs) address this performance trade-off by enabling smaller devices to offload part of their inference tasks to more capable devices. These systems often deploy hierarchical models that share numerous parameters, exemplified by Deep Neural Networks (DNNs) that utilize strategies like early exits or ordered dropout. In such instances, Federated Learning (FL) may be employed to jointly train the models within a CIS. Yet, traditional training methods have overlooked the operational dynamics of CISs during inference, particularly the potential high heterogeneity in serving rates across clients. To address this gap, we propose a novel FL approach designed explicitly for use in CISs that accounts for these variations in serving rates. Our framework not only offers rigorous theoretical guarantees, but also surpasses state-of-the-art (SOTA) training algorithms for CISs, especially in scenarios where inference request rates or data availability are uneven among clients.