A New Approach of Data Pre-processing for Data Compression in Smart Grids

TL;DR

This paper proposes a novel, utility-aware data pre-processing approach for smart grid data compression, improving the relevance of compressed data for decision-making by reducing the impact of noise.

Contribution

It introduces a decision-oriented pre-processing method tailored to utility functions, outperforming traditional transform-based techniques in smart grid applications.

Findings

Decision-oriented transforms reduce compression noise impact.

Linear and non-linear transforms outperform conventional methods.

Enhanced data utility in smart grid decision-making.

Abstract

The conventional approach to pre-process data for compression is to apply transforms such as the Fourier, the Karhunen-Lo\`{e}ve, or wavelet transforms. One drawback from adopting such an approach is that it is independent of the use of the compressed data, which may induce significant optimality losses when measured in terms of final utility (instead of being measured in terms of distortion). We therefore revisit this paradigm by tayloring the data pre-processing operation to the utility function of the decision-making entity using the compressed (and therefore noisy) data. More specifically, the utility function consists of an Lp-norm, which is very relevant in the area of smart grids. Both a linear and a non-linear use-oriented transforms are designed and compared with conventional data pre-processing techniques, showing that the impact of compression noise can be significantly reduced.