Optimal and Low-Complexity Dynamic Spectrum Access for RF-Powered Ambient Backscatter System with Online Reinforcement Learning

TL;DR

This paper introduces an optimal, low-complexity dynamic spectrum access method for RF-powered ambient backscatter systems, utilizing online reinforcement learning to adapt to environmental uncertainties and significantly improve system throughput and latency.

Contribution

It proposes a novel reinforcement learning-based approach for spectrum access in ambient backscatter systems, overcoming the need for complete environment knowledge.

Findings

Reinforcement learning improves throughput by up to 50%.

Reduces blocking probability and delay by up to 80%.

Efficiently adapts to environmental dynamics.

Abstract

Ambient backscatter has been introduced with a wide range of applications for low power wireless communications. In this article, we propose an optimal and low-complexity dynamic spectrum access framework for RF-powered ambient backscatter system. In this system, the secondary transmitter not only harvests energy from ambient signals (from incumbent users), but also backscatters these signals to its receiver for data transmission. Under the dynamics of the ambient signals, we first adopt the Markov decision process (MDP) framework to obtain the optimal policy for the secondary transmitter, aiming to maximize the system throughput. However, the MDP-based optimization requires complete knowledge of environment parameters, e.g., the probability of a channel to be idle and the probability of a successful packet transmission, that may not be practical to obtain. To cope with such incomplete knowledge of the environment, we develop a low-complexity online reinforcement learning algorithm that allows the secondary transmitter to "learn" from its decisions and then attain the optimal policy. Simulation results show that the proposed learning algorithm not only efficiently deals with the dynamics of the environment, but also improves the average throughput up to 50% and reduces the blocking probability and delay up to 80% compared with conventional methods.