Proactive QoE Provisioning in Heterogeneous Access Networks using Hidden Markov Models and Reinforcement Learning

TL;DR

This paper introduces a novel passive probing method using hidden Markov models for accurate QoE prediction and a reinforcement learning approach for proactive handoffs in heterogeneous networks, significantly reducing handoffs while maintaining high user experience.

Contribution

It presents a new passive probing technique for QoE prediction and a reinforcement learning framework for proactive handoff decisions, improving efficiency and user experience in heterogeneous networks.

Findings

QoE prediction accuracy of 97% using passive probing.

60.65% reduction in vertical handoffs.

Enhanced passive probing mechanisms for better network management.

Abstract

Quality of Experience (QoE) provisioning in heterogeneous access networks (HANs) can be achieved via handoffs. The current approaches for QoE-aware handoffs either lack the availability of a network path probing method or lack the availability of efficient methods for QoE prediction. Further, the current approaches do not explore the benefits of proactive QoE-aware handoffs such that user's QoE is maximized by learning from past network conditions and by actions taken by the mobile device regarding handoffs. In this paper, our contributions are two-fold. First, we propose, develop and validate a novel method for QoE prediction based on passive probing. Our method is based on hidden Markov models and Multi-homed Mobility Management Protocol which eliminates the need for additional probe packets for QoE prediction. It achieves the average QoE prediction accuracy of 97%. Second, we propose, develop and validate a novel reinforcement learning based method for proactive QoE-aware handoffs. We show that our method outperforms existing approaches by reducing the number of vertical handoffs by 60.65% while maintaining high QoE levels and by extending crucial functionality such as passive probing mechanisms.