IoTDataBench: Extending TPCx-IoT for Compression and Scalability

TL;DR

This paper introduces IoTDataBench, an extension of TPCx-IoT, to better evaluate IoT databases on data compression and scalability, highlighting the importance of these features for real-world IoT applications.

Contribution

It extends the TPCx-IoT benchmark to include data compression and scalability, providing a more comprehensive evaluation framework for IoT data management systems.

Findings

Systems with high compression ratios improve final metrics.

Linear scalability correlates with cost savings.

Effective compression and scalability are crucial for real-world IoT applications.

Abstract

We present a record-breaking result and lessons learned in practicing TPCx-IoT benchmarking for a real-world use case. We find that more system characteristics need to be benchmarked for its application to real-world use cases. We introduce an extension to the TPCx-IoT benchmark, covering fundamental requirements of time-series data management for IoT infrastructure. We characterize them as data compression and system scalability. To evaluate these two important features of IoT databases, we propose IoTDataBench and update four aspects of TPCx-IoT, i.e., data generation, workloads, metrics and test procedures. Preliminary evaluation results show systems that fail to effectively compress data or flexibly scale can negatively affect the redesigned metrics, while systems with high compression ratios and linear scalability are rewarded in the final metrics. Such systems have the ability to scale up computing resources on demand and can thus save dollar costs.