The Value of Information and Efficient Switching in Channel Selection

TL;DR

This paper analyzes optimal channel selection strategies in Markovian environments with switching costs, comparing full and partial observation scenarios, and characterizes optimal policies for specific cases.

Contribution

It introduces explicit optimal threshold policies for partial observation cases with two or infinitely many channels, highlighting the impact of information and switching costs.

Findings

Optimal threshold policies are characterized for partial observation scenarios.

Performance differences between full and partial observation are analyzed.

Switching costs significantly influence channel selection strategies.

Abstract

We consider a collection of statistically identical two-state continuous time Markov chains (channels). A controller continuously selects a channel with the view of maximizing infinite horizon average reward. A switching cost is paid upon channel changes. We consider two cases: full observation (all channels observed simultaneously) and partial observation (only the current channel observed). We analyze the difference in performance between these cases for various policies. For the partial observation case with two channels, or an infinite number of channels, we explicitly characterize an optimal threshold for two sensible policies which we name "call-gapping" and "cool-off". Our results present a qualitative view on the interaction of the number of channels, the available information, and the switching costs.