The Impact of Data Compression in Real-Time and Historical Data Acquisition Systems on the Accuracy of Analytical Solutions
Reham Faqehi, Haya Alhuraib, Hamad Saiari, Zyad Bamigdad

TL;DR
This paper investigates how data compression in industrial IoT systems affects the accuracy of analytics, highlighting trade-offs and proposing best practices for balancing efficiency and reliability.
Contribution
It provides a comprehensive analysis of compression impacts on analytical accuracy, combining theoretical, simulated, and empirical methods to guide optimal data management strategies.
Findings
Excessive compression can obscure critical data patterns.
Compression may skew statistical analysis results.
Optimal compression balances efficiency and analytical integrity.
Abstract
In industrial and IoT environments, massive amounts of real-time and historical process data are continuously generated and archived. With sensors and devices capturing every operational detail, the volume of time-series data has become a critical challenge for storage and processing systems. Efficient data management is essential to ensure scalability, cost-effectiveness, and timely analytics. To minimize storage expenses and optimize performance, data compression algorithms are frequently utilized in data historians and acquisition systems. However, compression comes with trade-offs that may compromise the accuracy and reliability of engineering analytics that depend on this compressed data. Understanding these trade-offs is essential for developing data strategies that support both operational efficiency and accurate, reliable analytics. This paper assesses the relation of common…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsTime Series Analysis and Forecasting · Advanced Data Storage Technologies · Algorithms and Data Compression
