Energy-Efficient and Federated Meta-Learning via Projected Stochastic Gradient Ascent

TL;DR

This paper introduces an energy-efficient federated meta-learning framework that uses projected stochastic gradient ascent to train a meta-model with minimal computation and communication energy, suitable for distributed tasks.

Contribution

It presents a lightweight algorithm that avoids complex computations like Hessian and matrix inversion, reducing energy use while maintaining high performance.

Findings

Achieves comparable or better performance than MAML and iMAML.

Significantly reduces energy consumption in federated meta-learning.

Effective on sinusoid regression and image classification tasks.

Abstract

In this paper, we propose an energy-efficient federated meta-learning framework. The objective is to enable learning a meta-model that can be fine-tuned to a new task with a few number of samples in a distributed setting and at low computation and communication energy consumption. We assume that each task is owned by a separate agent, so a limited number of tasks is used to train a meta-model. Assuming each task was trained offline on the agent's local data, we propose a lightweight algorithm that starts from the local models of all agents, and in a backward manner using projected stochastic gradient ascent (P-SGA) finds a meta-model. The proposed method avoids complex computations such as computing hessian, double looping, and matrix inversion, while achieving high performance at significantly less energy consumption compared to the state-of-the-art methods such as MAML and iMAML on conducted experiments for sinusoid regression and image classification tasks.