CAFEDistill: Learning Personalized and Dynamic Models through Federated Early-Exit Network Distillation

TL;DR

CAFEDistill introduces a federated learning framework that enables personalized, efficient, and adaptive inference by integrating early-exit networks with conflict-aware distillation, outperforming existing methods in accuracy and resource savings.

Contribution

It proposes a novel conflict-aware distillation method for federated early-exit networks, enabling personalized, adaptive, and resource-efficient model training across heterogeneous clients.

Findings

Achieves 30.79%-46.86% reduction in inference costs.

Outperforms state-of-the-art methods in accuracy.

Effectively mitigates conflicts among model exits.

Abstract

Personalized Federated Learning (PFL) enables collaboratively model training on decentralized, heterogeneous data while tailoring them to each client's unique distribution. However, existing PFL methods produce static models with a fixed tradeoff between accuracy and efficiency, limiting their applicability in environments where inference requirements vary with contexts and resource availability. Early-exit networks (EENs) offer adaptive inference by attaching intermediate classifiers. Yet integrating them into PFL is challenging due to client-wise heterogeneity and depth-wise interference arising from conflicting exit objectives. Prior studies fail to resolve both conflicts simultaneously, leading to suboptimal performance. In this paper, we propose CAFEDistill, a Conflict-Aware Federated Exit Distillation framework that jointly addresses these conflicts and extends PFL to early-exit networks. Through a progressive, depth-prioritized student coordination mechanism, CAFEDistill mitigates interference among shallow and deep exits while allowing effective personalized knowledge transfer across clients. Furthermore, it reduces communication overhead via a client-decoupled formulation. Extensive evaluations show that CAFEDistill outperforms the state-of-the-arts, achieving higher accuracy and reducing inference costs by 30.79%-46.86%.