A Web of Things Architecture for Digital Twin Creation and Model-Based Reinforcement Control

TL;DR

This paper introduces a Web of Things-based architecture for creating digital twins of smart environments, enabling real-time data-driven reinforcement learning control that improves convergence speed and tuning precision.

Contribution

It proposes a novel WoT-based digital twin architecture integrated with deep reinforcement learning for smart environment control, tested in real deployment.

Findings

Digital twins enable faster DRL convergence.

Enhanced tuning of control models through digital twins.

Successful real-world deployment demonstration.

Abstract

Internet of Things (IoT) devices are available in a multitude of scenarios, and provide constant, contextual data which can be leveraged to automatically reconfigure and optimize smart environments. To realize this vision, Artificial Intelligence (AI) and deep learning techniques are usually employed, however they need large quantity of data which is often not feasible in IoT scenarios. Digital Twins (DTs) have recently emerged as an effective way to replicate physical entities in the digital domain, to allow for simulation and testing of models and services. In this paper, we present a novel architecture based on the emerging Web of Things (WoT) standard, which provides a DT of a smart environment and applies Deep Reinforcement Learning (DRL) techniques on real time data. We implement our system in a real deployment, and test it along with a legacy system. Our findings show that the benefits of having a digital twin, specifically for DRL models, allow for faster convergence and finer tuning.