On the Energy and Communication Efficiency Tradeoffs in Federated and Multi-Task Learning

TL;DR

This paper investigates the energy and communication tradeoffs in federated and multi-task learning, demonstrating that Model-Agnostic Meta-Learning can significantly reduce energy costs in distributed wireless networks.

Contribution

It provides the first analysis of energy costs in MTL driven by MAML in wireless networks, highlighting factors influencing energy efficiency and communication tradeoffs.

Findings

MAML reduces energy costs by at least 2 times compared to traditional methods.

Energy efficiency depends on uplink/downlink and sidelink communication efficiencies.

Optimal energy balance varies with communication channel characteristics.

Abstract

Recent advances in Federated Learning (FL) have paved the way towards the design of novel strategies for solving multiple learning tasks simultaneously, by leveraging cooperation among networked devices. Multi-Task Learning (MTL) exploits relevant commonalities across tasks to improve efficiency compared with traditional transfer learning approaches. By learning multiple tasks jointly, significant reduction in terms of energy footprints can be obtained. This article provides a first look into the energy costs of MTL processes driven by the Model-Agnostic Meta-Learning (MAML) paradigm and implemented in distributed wireless networks. The paper targets a clustered multi-task network setup where autonomous agents learn different but related tasks. The MTL process is carried out in two stages: the optimization of a meta-model that can be quickly adapted to learn new tasks, and a task-specific model adaptation stage where the learned meta-model is transferred to agents and tailored for a specific task. This work analyzes the main factors that influence the MTL energy balance by considering a multi-task Reinforcement Learning (RL) setup in a robotized environment. Results show that the MAML method can reduce the energy bill by at least 2 times compared with traditional approaches without inductive transfer. Moreover, it is shown that the optimal energy balance in wireless networks depends on uplink/downlink and sidelink communication efficiencies.