Nonparametric Distributed Learning Architecture for Big Data: Algorithm and Applications

TL;DR

This paper introduces MetaLP, a flexible distributed framework designed to perform scalable statistical inference on large, complex datasets without altering traditional modeling principles.

Contribution

The paper proposes MetaLP, a novel nonparametric distributed learning architecture that handles diverse data types efficiently for big data applications.

Findings

MetaLP enables scalable inference on large datasets.

It accommodates various data types seamlessly.

The framework maintains statistical integrity in distributed settings.

Abstract

Dramatic increases in the size and complexity of modern datasets have made traditional "centralized" statistical inference prohibitive. In addition to computational challenges associated with big data learning, the presence of numerous data types (e.g. discrete, continuous, categorical, etc.) makes automation and scalability difficult. A question of immediate concern is how to design a data-intensive statistical inference architecture without changing the basic statistical modeling principles developed for "small" data over the last century. To address this problem, we present MetaLP, a flexible, distributed statistical modeling framework.