Lifelong Learning for Minimizing Age of Information in Internet of Things Networks

TL;DR

This paper introduces a lifelong reinforcement learning approach with a UAV as a flying base station to help IoT devices adapt quickly to dynamic environments, minimizing their age of information while conserving energy.

Contribution

It proposes a novel lifelong reinforcement learning algorithm utilizing a shared knowledge base at the UAV to enhance IoT devices' adaptation speed in changing environments.

Findings

Algorithm converges 25-50% faster than baseline.

Shared knowledge base improves adaptation efficiency.

Effective in dynamic IoT environments.

Abstract

In this paper, a lifelong learning problem is studied for an Internet of Things (IoT) system. In the considered model, each IoT device aims to balance its information freshness and energy consumption tradeoff by controlling its computational resource allocation at each time slot under dynamic environments. An unmanned aerial vehicle (UAV) is deployed as a flying base station so as to enable the IoT devices to adapt to novel environments. To this end, a new lifelong reinforcement learning algorithm, used by the UAV, is proposed in order to adapt the operation of the devices at each visit by the UAV. By using the experience from previously visited devices and environments, the UAV can help devices adapt faster to future states of their environment. To do so, a knowledge base shared by all devices is maintained at the UAV. Simulation results show that the proposed algorithm can converge $25\%$ to $50\%$ faster than a policy gradient baseline algorithm that optimizes each device's decision making problem in isolation.