Robust Fusion Methods for Structured Big Data

TL;DR

This paper introduces a robust framework for fusing estimators from subsamples in Big Data, focusing on multivariate, functional, and clustering problems, enhancing scalability and robustness.

Contribution

It presents a novel divide-and-conquer approach for robust estimator fusion applicable to various data structures in Big Data contexts.

Findings

Effective fusion methods for multivariate location and scatter matrices.

Robust covariance operator estimation for functional data.

Improved clustering techniques for large-scale datasets.

Abstract

We address one of the important problems in Big Data, namely how to combine estimators from different subsamples by robust fusion procedures, when we are unable to deal with the whole sample. We propose a general framework based on the classic idea of `divide and conquer'. In particular we address in some detail the case of a multivariate location and scatter matrix, the covariance operator for functional data, and clustering problems.