Distributed Inexact Damped Newton Method: Data Partitioning and Load-Balancing

TL;DR

This paper improves a distributed inexact damped Newton method by enhancing data partitioning and load-balancing, leading to reduced communication, better scalability, and more efficient computation, demonstrated on a large 273GB dataset.

Contribution

It proposes modifications to the DiSCO algorithm to improve scalability, communication efficiency, and load-balancing in distributed optimization.

Findings

Reduced communication costs in distributed optimization.

Enhanced load-balancing improves computational efficiency.

Successful application on a large-scale 273GB dataset.

Abstract

In this paper we study inexact dumped Newton method implemented in a distributed environment. We start with an original DiSCO algorithm [Communication-Efficient Distributed Optimization of Self-Concordant Empirical Loss, Yuchen Zhang and Lin Xiao, 2015]. We will show that this algorithm may not scale well and propose an algorithmic modifications which will lead to less communications, better load-balancing and more efficient computation. We perform numerical experiments with an regularized empirical loss minimization instance described by a 273GB dataset.