A Unified Algorithmic Framework for Block-Structured Optimization Involving Big Data

TL;DR

This paper introduces the BSUM framework, a versatile algorithmic approach for big data optimization that unifies many existing methods and demonstrates practical efficiency in various applications.

Contribution

The paper proposes the BSUM framework, unifying multiple optimization algorithms for big data and analyzing its features, implementation, and performance.

Findings

BSUM encompasses methods like BCD, CCCP, BCPG, NMF, EM.

BSUM shows flexibility and efficiency in large-scale problems.

Practical applications demonstrate BSUM's effectiveness.

Abstract

This article presents a powerful algorithmic framework for big data optimization, called the Block Successive Upper bound Minimization (BSUM). The BSUM includes as special cases many well-known methods for analyzing massive data sets, such as the Block Coordinate Descent (BCD), the Convex-Concave Procedure (CCCP), the Block Coordinate Proximal Gradient (BCPG) method, the Nonnegative Matrix Factorization (NMF), the Expectation Maximization (EM) method and so on. In this article, various features and properties of the BSUM are discussed from the viewpoint of design flexibility, computational efficiency, parallel/distributed implementation and the required communication overhead. Illustrative examples from networking, signal processing and machine learning are presented to demonstrate the practical performance of the BSUM framework