Semi-supervised Deep Reinforcement Learning in Support of IoT and Smart City Services

TL;DR

This paper introduces a novel semi-supervised deep reinforcement learning model that combines labeled and unlabeled data using Variational Autoencoders, specifically applied to indoor localization in smart city environments, achieving significant performance improvements.

Contribution

It is the first to extend deep reinforcement learning to a semi-supervised paradigm using VAEs for IoT and smart city applications.

Findings

23% improvement in localization accuracy

67% increase in received rewards

Effective use of unlabeled data in DRL

Abstract

Smart services are an important element of the smart cities and the Internet of Things (IoT) ecosystems where the intelligence behind the services is obtained and improved through the sensory data. Providing a large amount of training data is not always feasible; therefore, we need to consider alternative ways that incorporate unlabeled data as well. In recent years, Deep reinforcement learning (DRL) has gained great success in several application domains. It is an applicable method for IoT and smart city scenarios where auto-generated data can be partially labeled by users' feedback for training purposes. In this paper, we propose a semi-supervised deep reinforcement learning model that fits smart city applications as it consumes both labeled and unlabeled data to improve the performance and accuracy of the learning agent. The model utilizes Variational Autoencoders (VAE) as the inference engine for generalizing optimal policies. To the best of our knowledge, the proposed model is the first investigation that extends deep reinforcement learning to the semi-supervised paradigm. As a case study of smart city applications, we focus on smart buildings and apply the proposed model to the problem of indoor localization based on BLE signal strength. Indoor localization is the main component of smart city services since people spend significant time in indoor environments. Our model learns the best action policies that lead to a close estimation of the target locations with an improvement of 23% in terms of distance to the target and at least 67% more received rewards compared to the supervised DRL model.