FedPromo: Federated Lightweight Proxy Models at the Edge Bring New Domains to Foundation Models
Matteo Caligiuri, Francesco Barbato, Donald Shenaj, Umberto Michieli, Pietro Zanuttigh

TL;DR
FedPromo introduces a federated learning framework that efficiently adapts large foundation models to new domains using lightweight proxy models, reducing client-side computation while preserving privacy.
Contribution
It proposes a two-stage approach combining server-side knowledge distillation and local training of compact models, enabling resource-efficient personalized domain adaptation.
Findings
Outperforms existing methods on five image classification benchmarks.
Reduces computational overhead on client devices significantly.
Maintains high performance and privacy in multi-domain federated learning.
Abstract
Federated Learning (FL) is an established paradigm for training deep learning models on decentralized data. However, as the size of the models grows, conventional FL approaches often require significant computational resources on client devices, which may not be feasible. We introduce FedPromo, a novel framework that enables efficient adaptation of large-scale foundation models stored on a central server to new domains encountered only by remote clients. Instead of directly training the large model on client devices, FedPromo optimizes lightweight proxy models via FL, significantly reducing computational overhead while maintaining privacy. Our method follows a two-stage process: first, server-side knowledge distillation aligns the representations of a large-scale foundation model (e.g., a transformer) with those of a compact counterpart (e.g., a CNN). Then, the compact model encoder is…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsPrivacy-Preserving Technologies in Data · Cryptography and Data Security · Big Data and Digital Economy
