New reconstruction and data processing methods for regression and interpolation analysis of multidimensional big data

TL;DR

This paper introduces two innovative local area predictor-based methods for regression, interpolation, and reconstruction of multidimensional big data, addressing limitations of existing techniques in accuracy, efficiency, and high-dimensional applicability.

Contribution

The paper proposes two new methods utilizing local predictors, improving accuracy, computational performance, and suitability for high-dimensional big data processing.

Findings

Methods demonstrate high accuracy in up to 100 dimensions.

Significant improvements in computational efficiency and resource usage.

Methods are suitable for parallel computing and high-dimensional data analysis.

Abstract

The problems of computational data processing involving regression, interpolation, reconstruction and imputation for multidimensional big datasets are becoming more important these days, because of the availability of data and their widely spread usage in business, technological, scientific and other applications. The existing methods often have limitations, which either do not allow, or make it difficult to accomplish many data processing tasks. The problems usually relate to algorithm accuracy, applicability, performance (computational and algorithmic), demands for computational resources, both in terms of power and memory, and difficulty working with high dimensions. Here, we propose a new concept and introduce two methods, which use local area predictors (input data) for finding outcomes. One method uses the gradient based approach, while the second one employs an introduced family of smooth approximating functions. The new methods are free from many drawbacks of existing approaches. They are practical, have very wide range of applicability, provide high accuracy, excellent computational performance, fit for parallel computing, and very well suited for processing high dimension big data. The methods also provide multidimensional outcome, when needed. We present numerical examples of up to one hundred dimensions, and report in detail performance characteristics and various properties of new methods.