Data Interpolation: An Efficient Sampling Alternative for Big Data Aggregation

TL;DR

This paper introduces efficient polynomial interpolation methods for large sensor datasets, capable of handling noise and adversarial data, to better summarize and analyze big data in sensor networks.

Contribution

It extends existing interpolation techniques to multidimensional, noisy, and Byzantine data, providing robust solutions for big data aggregation.

Findings

Effective interpolation methods for noisy and Byzantine data

Extension of Welch-Berlekamp and Arora-Khot techniques to multidimensional data

Robust data representation for sensor data analysis

Abstract

Given a large set of measurement sensor data, in order to identify a simple function that captures the essence of the data gathered by the sensors, we suggest representing the data by (spatial) functions, in particular by polynomials. Given a (sampled) set of values, we interpolate the datapoints to define a polynomial that would represent the data. The interpolation is challenging, since in practice the data can be noisy and even Byzantine, where the Byzantine data represents an adversarial value that is not limited to being close to the correct measured data. We present two solutions, one that extends the Welch-Berlekamp technique in the case of multidimensional data, and copes with discrete noise and Byzantine data, and the other based on Arora and Khot techniques, extending them in the case of multidimensional noisy and Byzantine data.