Distributed Bayesian Matrix Decomposition for Big Data Mining and Clustering

TL;DR

This paper introduces a distributed Bayesian matrix decomposition model designed for big data analytics, capable of handling heterogeneous noise, scaling efficiently, and addressing communication challenges in distributed systems.

Contribution

It proposes a novel distributed Bayesian matrix decomposition method with three implementation strategies and theoretical convergence analysis for big data mining and clustering.

Findings

Algorithms scale well to big data.

Achieves superior or competitive performance.

Effectively models heterogeneous noise.

Abstract

Matrix decomposition is one of the fundamental tools to discover knowledge from big data generated by modern applications. However, it is still inefficient or infeasible to process very big data using such a method in a single machine. Moreover, big data are often distributedly collected and stored on different machines. Thus, such data generally bear strong heterogeneous noise. It is essential and useful to develop distributed matrix decomposition for big data analytics. Such a method should scale up well, model the heterogeneous noise, and address the communication issue in a distributed system. To this end, we propose a distributed Bayesian matrix decomposition model (DBMD) for big data mining and clustering. Specifically, we adopt three strategies to implement the distributed computing including 1) the accelerated gradient descent, 2) the alternating direction method of multipliers (ADMM), and 3) the statistical inference. We investigate the theoretical convergence behaviors of these algorithms. To address the heterogeneity of the noise, we propose an optimal plug-in weighted average that reduces the variance of the estimation. Synthetic experiments validate our theoretical results, and real-world experiments show that our algorithms scale up well to big data and achieves superior or competing performance compared to other distributed methods.