# A Reinforcement Learning Approach for the Multichannel Rendezvous Problem

## TL;DR

This paper applies reinforcement learning to optimize multichannel rendezvous in cognitive radio networks, modeling channel states as hidden Markov processes and minimizing expected time-to-rendezvous through adaptive policies.

## Contribution

It introduces a reinforcement learning framework for dynamic blind rendezvous policies, addressing unobservable channel states and formulating the problem as an adversarial bandit.

## Key findings

- Reinforcement learning achieves low ETTR comparable to existing methods.
- The approach effectively learns channel selection probabilities without observing channel states.
- Experimental results validate the method's efficiency in diverse scenarios.

## Abstract

In this paper, we consider the multichannel rendezvous problem in cognitive radio networks (CRNs) where the probability that two users hopping on the same channel have a successful rendezvous is a function of channel states. The channel states are modelled by two-state Markov chains that have a good state and a bad state. These channel states are not observable by the users. For such a multichannel rendezvous problem, we are interested in finding the optimal policy to minimize the expected time-to-rendezvous (ETTR) among the class of {\em dynamic blind rendezvous policies}, i.e., at the $t^{th}$ time slot each user selects channel $i$ independently with probability $p_i(t)$, $i=1,2, \ldots, N$. By formulating such a multichannel rendezvous problem as an adversarial bandit problem, we propose using a reinforcement learning approach to learn the channel selection probabilities $p_i(t)$, $i=1,2, \ldots, N$. Our experimental results show that the reinforcement learning approach is very effective and yields comparable ETTRs when comparing to various approximation policies in the literature.