Deep Q-Learning Assisted Bandwidth Reservation for Multi-Operator Time-Sensitive Vehicular Networking

TL;DR

This paper introduces a novel Deep Q-Learning based approach for multi-operator bandwidth reservation in vehicular networks, significantly reducing costs and improving reliability under uncertainty.

Contribution

It proposes a multi-objective DDQN strategy for bandwidth reservation, addressing cost minimization and reliability in uncertain vehicular network scenarios.

Findings

40% reduction in bandwidth costs across scenarios

Effective handling of underbooked and overbooked reservations

Outperforms greedy and other deep RL methods

Abstract

Very few available individual bandwidth reservation schemes provide efficient and cost-effective bandwidth reservation that is required for safety-critical and time-sensitive vehicular networked applications. These schemes allow vehicles to make reservation requests for the required resources. Accordingly, a Mobile Network Operator (MNO) can allocate and guarantee bandwidth resources based on these requests. However, due to uncertainty in future reservation time and bandwidth costs, the design of an optimized reservation strategy is challenging. In this article, we propose a novel multi-objective bandwidth reservation update approach with an optimal strategy based on Double Deep Q-Network (DDQN). The key design objectives are to minimize the reservation cost with multiple MNOs and to ensure reliable resource provisioning in uncertain situations by solving scenarios such as underbooked and overbooked reservations along the driving path. The enhancements and advantages of our proposed strategy have been demonstrated through extensive experimental results when compared to other methods like greedy update or other deep reinforcement learning approaches. Our strategy demonstrates a 40% reduction in bandwidth costs across all investigated scenarios and simultaneously resolves uncertain situations in a cost-effective manner.